2023-08-24T09:51:17.364 INFO:root:teuthology version: 0.0.1.dev185+g449a1bc 2023-08-24T09:51:17.365 DEBUG:teuthology.run:Teuthology command: teuthology --description crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} --verbose --owner scheduled_matan@teuthology --archive /home/teuthworker/archive/matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi/7378498 --name matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi -- /home/teuthworker/archive/matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi/7378498/orig.config.yaml 2023-08-24T09:51:17.399 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-24T09:51:17.483 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi/7378498 branch: wip-matanb-crimson-testing-24.8-v2 description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} email: null first_in_suite: false job_id: '7378498' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-matanb-crimson-testing-24.8-v2 ceph: conf: client: debug ms: 1 global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd pool default crimson: true osd: crimson seastar smp: 3 debug ms: 1 debug osd: 20 osd blocked scrub grace period: 3600 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: crimson log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: crimson sha1: 74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe workunit: branch: wip-matanb-crimson-testing-24.8-v2 env: CRIMSON_COMPAT: '1' sha1: 74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe owner: scheduled_matan@teuthology priority: 60 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b sha1: 74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe sleep_before_teardown: 0 suite: crimson-rados suite_branch: wip-matanb-crimson-testing-24.8-v2 suite_path: /home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa suite_relpath: qa suite_repo: https://github.com/Matan-B/ceph.git suite_sha1: 74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe targets: smithi060.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN6gNUKYLjbh09BzRantQjCUQIPym0AcypVmYdTq0EQhIUlhNLQG2LbakWKji0BnsgZy+jdMt47mxajWjPvBTFc= smithi135.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLiGtAHJjeXgDa3WaBv86LmWFjgoBL171vZvf/ibHaA9UP1RgxslYuGqz4owpEQtVYvE+83r0dcAeW86D74iIBc= tasks: - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 flavor: crimson - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 449a1bc2027504e7b3c3d7b30fa4178906581da7 timestamp: 2023-08-24_09:25:19 tube: smithi user: matan verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2224 2023-08-24T09:51:17.483 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa; will attempt to use it 2023-08-24T09:51:17.484 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks 2023-08-24T09:51:17.485 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-08-24T09:51:17.487 INFO:teuthology.task.internal:Checking packages... 2023-08-24T09:51:17.522 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'crimson' and ceph hash '74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe' 2023-08-24T09:51:17.523 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-08-24T09:51:17.523 INFO:teuthology.packaging:ref: None 2023-08-24T09:51:17.523 INFO:teuthology.packaging:tag: None 2023-08-24T09:51:17.524 INFO:teuthology.packaging:branch: wip-matanb-crimson-testing-24.8-v2 2023-08-24T09:51:17.524 INFO:teuthology.packaging:sha1: 74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe 2023-08-24T09:51:17.524 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&ref=wip-matanb-crimson-testing-24.8-v2 2023-08-24T09:51:17.741 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-5670.g74f8c0ae 2023-08-24T09:51:17.742 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-08-24T09:51:17.748 INFO:teuthology.task.internal:no buildpackages task found 2023-08-24T09:51:17.748 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-08-24T09:51:17.764 INFO:teuthology.task.internal:Saving configuration 2023-08-24T09:51:17.777 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-08-24T09:51:17.784 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-24T09:51:17.808 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi/7378498', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-24 09:46:20.439239', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN6gNUKYLjbh09BzRantQjCUQIPym0AcypVmYdTq0EQhIUlhNLQG2LbakWKji0BnsgZy+jdMt47mxajWjPvBTFc='} 2023-08-24T09:51:17.834 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi/7378498', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-24 09:46:20.437484', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLiGtAHJjeXgDa3WaBv86LmWFjgoBL171vZvf/ibHaA9UP1RgxslYuGqz4owpEQtVYvE+83r0dcAeW86D74iIBc='} 2023-08-24T09:51:17.834 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-08-24T09:51:17.842 INFO:teuthology.task.internal:roles: ubuntu@smithi060.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2023-08-24T09:51:17.842 INFO:teuthology.task.internal:roles: ubuntu@smithi135.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-08-24T09:51:17.842 INFO:teuthology.run_tasks:Running task console_log... 2023-08-24T09:51:17.920 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f564f39bdc0>, signals=[15]) 2023-08-24T09:51:17.921 INFO:teuthology.run_tasks:Running task internal.connect... 2023-08-24T09:51:17.934 INFO:teuthology.task.internal:Opening connections... 2023-08-24T09:51:17.934 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi060.front.sepia.ceph.com 2023-08-24T09:51:17.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-24T09:51:18.038 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi135.front.sepia.ceph.com 2023-08-24T09:51:18.039 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-24T09:51:18.123 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-08-24T09:51:18.129 DEBUG:teuthology.orchestra.run.smithi060:> uname -m 2023-08-24T09:51:18.149 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64 2023-08-24T09:51:18.149 DEBUG:teuthology.orchestra.run.smithi060:> cat /etc/os-release 2023-08-24T09:51:18.206 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="CentOS Stream" 2023-08-24T09:51:18.206 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="9" 2023-08-24T09:51:18.206 INFO:teuthology.orchestra.run.smithi060.stdout:ID="centos" 2023-08-24T09:51:18.207 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE="rhel fedora" 2023-08-24T09:51:18.207 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="9" 2023-08-24T09:51:18.207 INFO:teuthology.orchestra.run.smithi060.stdout:PLATFORM_ID="platform:el9" 2023-08-24T09:51:18.207 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-24T09:51:18.207 INFO:teuthology.orchestra.run.smithi060.stdout:ANSI_COLOR="0;31" 2023-08-24T09:51:18.207 INFO:teuthology.orchestra.run.smithi060.stdout:LOGO="fedora-logo-icon" 2023-08-24T09:51:18.207 INFO:teuthology.orchestra.run.smithi060.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-24T09:51:18.208 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://centos.org/" 2023-08-24T09:51:18.208 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-24T09:51:18.208 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-24T09:51:18.208 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-24T09:51:18.209 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2023-08-24T09:51:18.233 DEBUG:teuthology.orchestra.run.smithi135:> uname -m 2023-08-24T09:51:18.251 INFO:teuthology.orchestra.run.smithi135.stdout:x86_64 2023-08-24T09:51:18.251 DEBUG:teuthology.orchestra.run.smithi135:> cat /etc/os-release 2023-08-24T09:51:18.307 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="CentOS Stream" 2023-08-24T09:51:18.308 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="9" 2023-08-24T09:51:18.308 INFO:teuthology.orchestra.run.smithi135.stdout:ID="centos" 2023-08-24T09:51:18.308 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE="rhel fedora" 2023-08-24T09:51:18.308 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="9" 2023-08-24T09:51:18.308 INFO:teuthology.orchestra.run.smithi135.stdout:PLATFORM_ID="platform:el9" 2023-08-24T09:51:18.309 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-24T09:51:18.309 INFO:teuthology.orchestra.run.smithi135.stdout:ANSI_COLOR="0;31" 2023-08-24T09:51:18.309 INFO:teuthology.orchestra.run.smithi135.stdout:LOGO="fedora-logo-icon" 2023-08-24T09:51:18.309 INFO:teuthology.orchestra.run.smithi135.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-24T09:51:18.309 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://centos.org/" 2023-08-24T09:51:18.309 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-24T09:51:18.309 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-24T09:51:18.310 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-24T09:51:18.310 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2023-08-24T09:51:18.332 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-08-24T09:51:18.342 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-08-24T09:51:18.348 INFO:teuthology.task.internal:Checking for old test directory... 2023-08-24T09:51:18.349 DEBUG:teuthology.orchestra.run.smithi060:> test '!' -e /home/ubuntu/cephtest 2023-08-24T09:51:18.351 DEBUG:teuthology.orchestra.run.smithi135:> test '!' -e /home/ubuntu/cephtest 2023-08-24T09:51:18.367 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-08-24T09:51:18.373 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-08-24T09:51:18.374 DEBUG:teuthology.orchestra.run.smithi060:> test -z $(ls -A /var/lib/ceph) 2023-08-24T09:51:18.408 DEBUG:teuthology.orchestra.run.smithi135:> test -z $(ls -A /var/lib/ceph) 2023-08-24T09:51:18.441 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-08-24T09:51:18.492 INFO:teuthology.run_tasks:Running task kernel... 2023-08-24T09:51:18.521 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-08-24T09:51:18.522 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-08-24T09:51:18.522 DEBUG:teuthology.orchestra.run.smithi060:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-24T09:51:18.523 DEBUG:teuthology.orchestra.run.smithi135:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-24T09:51:18.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-24T09:51:18.542 DEBUG:teuthology.orchestra.run.smithi135:> uname -r 2023-08-24T09:51:18.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-24T09:51:18.544 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2023-08-24T09:51:18.598 INFO:teuthology.orchestra.run.smithi135.stdout:5.14.0-239.el9.x86_64 2023-08-24T09:51:18.598 INFO:teuthology.task.kernel:Running kernel on smithi135: 5.14.0-239.el9.x86_64 2023-08-24T09:51:18.598 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum install -y kernel 2023-08-24T09:51:18.601 INFO:teuthology.orchestra.run.smithi060.stdout:5.14.0-239.el9.x86_64 2023-08-24T09:51:18.601 INFO:teuthology.task.kernel:Running kernel on smithi060: 5.14.0-239.el9.x86_64 2023-08-24T09:51:18.602 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum install -y kernel 2023-08-24T09:51:20.795 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS-9 - AppStream 8.8 MB/s | 6.2 MB 00:00 2023-08-24T09:51:20.883 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS-9 - AppStream 11 MB/s | 6.2 MB 00:00 2023-08-24T09:51:22.079 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS-9 - BaseOS 7.1 MB/s | 2.0 MB 00:00 2023-08-24T09:51:22.258 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS-9 - BaseOS 6.1 MB/s | 2.0 MB 00:00 2023-08-24T09:51:24.620 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - BaseOS 3.5 MB/s | 7.8 MB 00:02 2023-08-24T09:51:26.682 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 7.8 MB 00:04 2023-08-24T09:51:28.475 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 23 MB/s | 18 MB 00:00 2023-08-24T09:51:30.137 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - AppStream 3.9 MB/s | 18 MB 00:04 2023-08-24T09:51:32.526 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 14 kB 00:00 2023-08-24T09:51:33.639 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 14 kB 00:00 2023-08-24T09:51:34.170 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 12 MB/s | 18 MB 00:01 2023-08-24T09:51:34.531 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 23 MB/s | 18 MB 00:00 2023-08-24T09:51:40.225 INFO:teuthology.orchestra.run.smithi060.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-08-24T09:51:40.314 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T09:51:40.324 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T09:51:40.324 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2023-08-24T09:51:40.324 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T09:51:40.324 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-08-24T09:51:40.325 INFO:teuthology.orchestra.run.smithi060.stdout: kernel x86_64 5.14.0-359.el9 CentOS-BaseOS 5.0 M 2023-08-24T09:51:40.325 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2023-08-24T09:51:40.325 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-core x86_64 5.14.0-359.el9 CentOS-BaseOS 20 M 2023-08-24T09:51:40.325 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules x86_64 5.14.0-359.el9 CentOS-BaseOS 37 M 2023-08-24T09:51:40.325 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules-core x86_64 5.14.0-359.el9 CentOS-BaseOS 32 M 2023-08-24T09:51:40.326 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T09:51:40.326 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-08-24T09:51:40.326 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T09:51:40.326 INFO:teuthology.orchestra.run.smithi060.stdout:Install 4 Packages 2023-08-24T09:51:40.326 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T09:51:40.327 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 93 M 2023-08-24T09:51:40.327 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 120 M 2023-08-24T09:51:40.328 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-08-24T09:51:40.361 INFO:teuthology.orchestra.run.smithi135.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-08-24T09:51:40.467 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T09:51:40.477 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T09:51:40.477 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-08-24T09:51:40.478 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T09:51:40.478 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-08-24T09:51:40.478 INFO:teuthology.orchestra.run.smithi135.stdout: kernel x86_64 5.14.0-359.el9 CentOS-BaseOS 5.0 M 2023-08-24T09:51:40.478 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-08-24T09:51:40.478 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-core x86_64 5.14.0-359.el9 CentOS-BaseOS 20 M 2023-08-24T09:51:40.478 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules x86_64 5.14.0-359.el9 CentOS-BaseOS 37 M 2023-08-24T09:51:40.479 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules-core x86_64 5.14.0-359.el9 CentOS-BaseOS 32 M 2023-08-24T09:51:40.479 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T09:51:40.479 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-08-24T09:51:40.479 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T09:51:40.479 INFO:teuthology.orchestra.run.smithi135.stdout:Install 4 Packages 2023-08-24T09:51:40.479 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T09:51:40.479 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 93 M 2023-08-24T09:51:40.479 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 120 M 2023-08-24T09:51:40.480 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-08-24T09:51:40.921 INFO:teuthology.orchestra.run.smithi060.stdout:(1/4): kernel-5.14.0-359.el9.x86_64.rpm 8.5 MB/s | 5.0 MB 00:00 2023-08-24T09:51:41.076 INFO:teuthology.orchestra.run.smithi135.stdout:(1/4): kernel-5.14.0-359.el9.x86_64.rpm 8.5 MB/s | 5.0 MB 00:00 2023-08-24T09:51:41.889 INFO:teuthology.orchestra.run.smithi060.stdout:(2/4): kernel-core-5.14.0-359.el9.x86_64.rpm 13 MB/s | 20 MB 00:01 2023-08-24T09:51:41.923 INFO:teuthology.orchestra.run.smithi135.stdout:(2/4): kernel-core-5.14.0-359.el9.x86_64.rpm 14 MB/s | 20 MB 00:01 2023-08-24T09:51:42.642 INFO:teuthology.orchestra.run.smithi135.stdout:(3/4): kernel-modules-core-5.14.0-359.el9.x86_6 20 MB/s | 32 MB 00:01 2023-08-24T09:51:42.988 INFO:teuthology.orchestra.run.smithi060.stdout:(3/4): kernel-modules-core-5.14.0-359.el9.x86_6 15 MB/s | 32 MB 00:02 2023-08-24T09:51:43.324 INFO:teuthology.orchestra.run.smithi135.stdout:(4/4): kernel-modules-5.14.0-359.el9.x86_64.rpm 13 MB/s | 37 MB 00:02 2023-08-24T09:51:43.327 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-08-24T09:51:43.327 INFO:teuthology.orchestra.run.smithi135.stdout:Total 33 MB/s | 93 MB 00:02 2023-08-24T09:51:43.328 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-08-24T09:51:43.479 INFO:teuthology.orchestra.run.smithi060.stdout:(4/4): kernel-modules-5.14.0-359.el9.x86_64.rpm 12 MB/s | 37 MB 00:03 2023-08-24T09:51:43.482 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-08-24T09:51:43.482 INFO:teuthology.orchestra.run.smithi060.stdout:Total 30 MB/s | 93 MB 00:03 2023-08-24T09:51:43.483 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-08-24T09:51:43.654 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-08-24T09:51:43.654 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-08-24T09:51:43.890 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-08-24T09:51:43.890 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-08-24T09:51:44.489 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-08-24T09:51:44.489 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-08-24T09:51:44.734 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-08-24T09:51:44.734 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-08-24T09:51:46.681 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-08-24T09:51:47.198 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-08-24T09:51:47.318 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-modules-core-5.14.0-359.el9.x86_64 1/4 2023-08-24T09:51:47.458 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-core-5.14.0-359.el9.x86_64 2/4 2023-08-24T09:51:47.887 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-modules-core-5.14.0-359.el9.x86_64 1/4 2023-08-24T09:51:48.029 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-core-5.14.0-359.el9.x86_64 2/4 2023-08-24T09:51:49.680 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-core-5.14.0-359.el9.x86_64 2/4 2023-08-24T09:51:49.857 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-modules-5.14.0-359.el9.x86_64 3/4 2023-08-24T09:51:50.423 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-core-5.14.0-359.el9.x86_64 2/4 2023-08-24T09:51:50.619 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-modules-5.14.0-359.el9.x86_64 3/4 2023-08-24T09:51:54.401 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-modules-5.14.0-359.el9.x86_64 3/4 2023-08-24T09:51:54.599 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-5.14.0-359.el9.x86_64 4/4 2023-08-24T09:51:55.560 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-modules-5.14.0-359.el9.x86_64 3/4 2023-08-24T09:51:55.704 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-5.14.0-359.el9.x86_64 4/4 2023-08-24T09:51:59.185 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-modules-core-5.14.0-359.el9.x86_64 4/4 2023-08-24T09:52:00.658 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-modules-core-5.14.0-359.el9.x86_64 4/4 2023-08-24T09:52:28.451 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-core-5.14.0-359.el9.x86_64 4/4 2023-08-24T09:52:28.492 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-modules-5.14.0-359.el9.x86_64 4/4 2023-08-24T09:52:29.088 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-5.14.0-359.el9.x86_64 4/4 2023-08-24T09:52:29.088 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-5.14.0-359.el9.x86_64 1/4 2023-08-24T09:52:29.089 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-core-5.14.0-359.el9.x86_64 2/4 2023-08-24T09:52:29.090 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-modules-5.14.0-359.el9.x86_64 3/4 2023-08-24T09:52:29.539 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-modules-core-5.14.0-359.el9.x86_64 4/4 2023-08-24T09:52:29.539 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T09:52:29.539 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-08-24T09:52:29.540 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-5.14.0-359.el9.x86_64 2023-08-24T09:52:29.540 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-core-5.14.0-359.el9.x86_64 2023-08-24T09:52:29.540 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules-5.14.0-359.el9.x86_64 2023-08-24T09:52:29.541 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules-core-5.14.0-359.el9.x86_64 2023-08-24T09:52:29.541 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T09:52:29.541 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T09:52:30.064 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q kernel | sort -rV | head -n 1 2023-08-24T09:52:30.165 INFO:teuthology.orchestra.run.smithi135.stdout:kernel-5.14.0-359.el9.x86_64 2023-08-24T09:52:30.166 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-359.el9.x86_64 2023-08-24T09:52:30.166 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-239.el9.x86_64 Expected: 5.14.0-359.el9.x86_64 2023-08-24T09:52:30.166 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-08-24T09:52:30.166 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-08-24T09:52:30.167 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2023-08-24T09:52:30.167 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi135.front.sepia.ceph.com, path=None, version=distro) 2023-08-24T09:52:30.167 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q kernel | sort -rV | head -n 1 2023-08-24T09:52:30.236 INFO:teuthology.orchestra.run.smithi135.stdout:kernel-5.14.0-359.el9.x86_64 2023-08-24T09:52:30.236 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-359.el9.x86_64 2023-08-24T09:52:30.236 DEBUG:teuthology.orchestra.run.smithi135:> sudo rpm -qi grub2-tools 2023-08-24T09:52:30.282 INFO:teuthology.orchestra.run.smithi135.stdout:Name : grub2-tools 2023-08-24T09:52:30.282 INFO:teuthology.orchestra.run.smithi135.stdout:Epoch : 1 2023-08-24T09:52:30.282 INFO:teuthology.orchestra.run.smithi135.stdout:Version : 2.06 2023-08-24T09:52:30.282 INFO:teuthology.orchestra.run.smithi135.stdout:Release : 53.el9 2023-08-24T09:52:30.283 INFO:teuthology.orchestra.run.smithi135.stdout:Architecture: x86_64 2023-08-24T09:52:30.283 INFO:teuthology.orchestra.run.smithi135.stdout:Install Date: Mon 30 Jan 2023 10:38:40 AM UTC 2023-08-24T09:52:30.283 INFO:teuthology.orchestra.run.smithi135.stdout:Group : Unspecified 2023-08-24T09:52:30.283 INFO:teuthology.orchestra.run.smithi135.stdout:Size : 8551923 2023-08-24T09:52:30.284 INFO:teuthology.orchestra.run.smithi135.stdout:License : GPLv3+ 2023-08-24T09:52:30.284 INFO:teuthology.orchestra.run.smithi135.stdout:Signature : RSA/SHA256, Wed 18 Jan 2023 04:40:07 PM UTC, Key ID 05b555b38483c65d 2023-08-24T09:52:30.284 INFO:teuthology.orchestra.run.smithi135.stdout:Source RPM : grub2-2.06-53.el9.src.rpm 2023-08-24T09:52:30.284 INFO:teuthology.orchestra.run.smithi135.stdout:Build Date : Fri 13 Jan 2023 09:07:28 PM UTC 2023-08-24T09:52:30.284 INFO:teuthology.orchestra.run.smithi135.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-08-24T09:52:30.284 INFO:teuthology.orchestra.run.smithi135.stdout:Packager : builder@centos.org 2023-08-24T09:52:30.284 INFO:teuthology.orchestra.run.smithi135.stdout:Vendor : CentOS 2023-08-24T09:52:30.285 INFO:teuthology.orchestra.run.smithi135.stdout:URL : http://www.gnu.org/software/grub/ 2023-08-24T09:52:30.285 INFO:teuthology.orchestra.run.smithi135.stdout:Summary : Support tools for GRUB. 2023-08-24T09:52:30.285 INFO:teuthology.orchestra.run.smithi135.stdout:Description : 2023-08-24T09:52:30.285 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T09:52:30.285 INFO:teuthology.orchestra.run.smithi135.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-08-24T09:52:30.285 INFO:teuthology.orchestra.run.smithi135.stdout:customizable bootloader with modular architecture. It supports a rich 2023-08-24T09:52:30.285 INFO:teuthology.orchestra.run.smithi135.stdout:variety of kernel formats, file systems, computer architectures and 2023-08-24T09:52:30.285 INFO:teuthology.orchestra.run.smithi135.stdout:hardware devices. 2023-08-24T09:52:30.286 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T09:52:30.286 INFO:teuthology.orchestra.run.smithi135.stdout:This subpackage provides tools for support of all platforms. 2023-08-24T09:52:30.288 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-08-24T09:52:30.288 INFO:teuthology.task.kernel:Updating grub on smithi135 to boot 5.14.0-359.el9.x86_64 2023-08-24T09:52:30.288 DEBUG:teuthology.orchestra.run.smithi135:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-08-24T09:52:30.960 INFO:teuthology.orchestra.run.smithi135.stderr:Generating grub configuration file ... 2023-08-24T09:52:32.244 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-core-5.14.0-359.el9.x86_64 4/4 2023-08-24T09:52:32.306 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-modules-5.14.0-359.el9.x86_64 4/4 2023-08-24T09:52:32.404 INFO:teuthology.orchestra.run.smithi135.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-08-24T09:52:32.438 INFO:teuthology.orchestra.run.smithi135.stderr:done 2023-08-24T09:52:32.449 DEBUG:teuthology.orchestra.run.smithi135:> mktemp 2023-08-24T09:52:32.505 INFO:teuthology.orchestra.run.smithi135.stdout:/tmp/tmp.qcNedfF9lB 2023-08-24T09:52:32.505 DEBUG:teuthology.orchestra.run.smithi135:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.qcNedfF9lB 2023-08-24T09:52:32.590 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0666 /tmp/tmp.qcNedfF9lB 2023-08-24T09:52:32.732 DEBUG:teuthology.orchestra.remote:smithi135:/tmp/tmp.qcNedfF9lB is 6KB 2023-08-24T09:52:32.781 DEBUG:teuthology.orchestra.run.smithi135:> rm -fr /tmp/tmp.qcNedfF9lB 2023-08-24T09:52:32.795 DEBUG:teuthology.orchestra.run.smithi135:> sudo /bin/ls /boot/loader/entries || true 2023-08-24T09:52:32.859 INFO:teuthology.orchestra.run.smithi135.stdout:e13cba1d25f14c51a1d04da8db823a4e-0-rescue.conf 2023-08-24T09:52:32.859 INFO:teuthology.orchestra.run.smithi135.stdout:e13cba1d25f14c51a1d04da8db823a4e-5.14.0-239.el9.x86_64.conf 2023-08-24T09:52:32.859 INFO:teuthology.orchestra.run.smithi135.stdout:e13cba1d25f14c51a1d04da8db823a4e-5.14.0-359.el9.x86_64.conf 2023-08-24T09:52:32.861 DEBUG:teuthology.orchestra.run.smithi135:> sudo grub2-set-default e13cba1d25f14c51a1d04da8db823a4e-5.14.0-359.el9.x86_64 2023-08-24T09:52:32.892 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-5.14.0-359.el9.x86_64 4/4 2023-08-24T09:52:32.892 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-5.14.0-359.el9.x86_64 1/4 2023-08-24T09:52:32.892 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-core-5.14.0-359.el9.x86_64 2/4 2023-08-24T09:52:32.892 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-modules-5.14.0-359.el9.x86_64 3/4 2023-08-24T09:52:33.020 DEBUG:teuthology.orchestra.run.smithi135:> sudo shutdown -r now 2023-08-24T09:52:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-modules-core-5.14.0-359.el9.x86_64 4/4 2023-08-24T09:52:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T09:52:33.395 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-08-24T09:52:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-5.14.0-359.el9.x86_64 2023-08-24T09:52:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-core-5.14.0-359.el9.x86_64 2023-08-24T09:52:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules-5.14.0-359.el9.x86_64 2023-08-24T09:52:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules-core-5.14.0-359.el9.x86_64 2023-08-24T09:52:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T09:52:33.396 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T09:52:34.119 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q kernel | sort -rV | head -n 1 2023-08-24T09:52:34.221 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-5.14.0-359.el9.x86_64 2023-08-24T09:52:34.221 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-359.el9.x86_64 2023-08-24T09:52:34.221 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-239.el9.x86_64 Expected: 5.14.0-359.el9.x86_64 2023-08-24T09:52:34.222 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-08-24T09:52:34.222 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-08-24T09:52:34.222 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-08-24T09:52:34.222 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi060.front.sepia.ceph.com, path=None, version=distro) 2023-08-24T09:52:34.222 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q kernel | sort -rV | head -n 1 2023-08-24T09:52:34.294 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-5.14.0-359.el9.x86_64 2023-08-24T09:52:34.295 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-359.el9.x86_64 2023-08-24T09:52:34.295 DEBUG:teuthology.orchestra.run.smithi060:> sudo rpm -qi grub2-tools 2023-08-24T09:52:34.345 INFO:teuthology.orchestra.run.smithi060.stdout:Name : grub2-tools 2023-08-24T09:52:34.366 INFO:teuthology.orchestra.run.smithi060.stdout:Epoch : 1 2023-08-24T09:52:34.366 INFO:teuthology.orchestra.run.smithi060.stdout:Version : 2.06 2023-08-24T09:52:34.367 INFO:teuthology.orchestra.run.smithi060.stdout:Release : 53.el9 2023-08-24T09:52:34.367 INFO:teuthology.orchestra.run.smithi060.stdout:Architecture: x86_64 2023-08-24T09:52:34.367 INFO:teuthology.orchestra.run.smithi060.stdout:Install Date: Mon 30 Jan 2023 10:38:40 AM UTC 2023-08-24T09:52:34.368 INFO:teuthology.orchestra.run.smithi060.stdout:Group : Unspecified 2023-08-24T09:52:34.368 INFO:teuthology.orchestra.run.smithi060.stdout:Size : 8551923 2023-08-24T09:52:34.368 INFO:teuthology.orchestra.run.smithi060.stdout:License : GPLv3+ 2023-08-24T09:52:34.368 INFO:teuthology.orchestra.run.smithi060.stdout:Signature : RSA/SHA256, Wed 18 Jan 2023 04:40:07 PM UTC, Key ID 05b555b38483c65d 2023-08-24T09:52:34.368 INFO:teuthology.orchestra.run.smithi060.stdout:Source RPM : grub2-2.06-53.el9.src.rpm 2023-08-24T09:52:34.368 INFO:teuthology.orchestra.run.smithi060.stdout:Build Date : Fri 13 Jan 2023 09:07:28 PM UTC 2023-08-24T09:52:34.369 INFO:teuthology.orchestra.run.smithi060.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-08-24T09:52:34.369 INFO:teuthology.orchestra.run.smithi060.stdout:Packager : builder@centos.org 2023-08-24T09:52:34.369 INFO:teuthology.orchestra.run.smithi060.stdout:Vendor : CentOS 2023-08-24T09:52:34.369 INFO:teuthology.orchestra.run.smithi060.stdout:URL : http://www.gnu.org/software/grub/ 2023-08-24T09:52:34.369 INFO:teuthology.orchestra.run.smithi060.stdout:Summary : Support tools for GRUB. 2023-08-24T09:52:34.369 INFO:teuthology.orchestra.run.smithi060.stdout:Description : 2023-08-24T09:52:34.370 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T09:52:34.370 INFO:teuthology.orchestra.run.smithi060.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-08-24T09:52:34.370 INFO:teuthology.orchestra.run.smithi060.stdout:customizable bootloader with modular architecture. It supports a rich 2023-08-24T09:52:34.370 INFO:teuthology.orchestra.run.smithi060.stdout:variety of kernel formats, file systems, computer architectures and 2023-08-24T09:52:34.370 INFO:teuthology.orchestra.run.smithi060.stdout:hardware devices. 2023-08-24T09:52:34.370 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T09:52:34.371 INFO:teuthology.orchestra.run.smithi060.stdout:This subpackage provides tools for support of all platforms. 2023-08-24T09:52:34.371 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-08-24T09:52:34.372 INFO:teuthology.task.kernel:Updating grub on smithi060 to boot 5.14.0-359.el9.x86_64 2023-08-24T09:52:34.372 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-08-24T09:52:35.301 INFO:teuthology.orchestra.run.smithi060.stderr:Generating grub configuration file ... 2023-08-24T09:52:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-08-24T09:52:36.887 INFO:teuthology.orchestra.run.smithi060.stderr:done 2023-08-24T09:52:36.889 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2023-08-24T09:52:36.936 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.mgVs7fEj8K 2023-08-24T09:52:36.937 DEBUG:teuthology.orchestra.run.smithi060:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.mgVs7fEj8K 2023-08-24T09:52:36.974 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.mgVs7fEj8K 2023-08-24T09:52:37.129 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.mgVs7fEj8K is 6KB 2023-08-24T09:52:37.151 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.mgVs7fEj8K 2023-08-24T09:52:37.165 DEBUG:teuthology.orchestra.run.smithi060:> sudo /bin/ls /boot/loader/entries || true 2023-08-24T09:52:37.234 INFO:teuthology.orchestra.run.smithi060.stdout:e13cba1d25f14c51a1d04da8db823a4e-0-rescue.conf 2023-08-24T09:52:37.234 INFO:teuthology.orchestra.run.smithi060.stdout:e13cba1d25f14c51a1d04da8db823a4e-5.14.0-239.el9.x86_64.conf 2023-08-24T09:52:37.235 INFO:teuthology.orchestra.run.smithi060.stdout:e13cba1d25f14c51a1d04da8db823a4e-5.14.0-359.el9.x86_64.conf 2023-08-24T09:52:37.235 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-set-default e13cba1d25f14c51a1d04da8db823a4e-5.14.0-359.el9.x86_64 2023-08-24T09:52:37.457 DEBUG:teuthology.orchestra.run.smithi060:> sudo shutdown -r now 2023-08-24T09:53:03.049 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2023-08-24T09:53:03.049 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2023-08-24T09:53:03.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-24T09:53:07.464 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-08-24T09:53:07.465 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2023-08-24T09:53:07.465 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-24T09:54:03.104 DEBUG:teuthology.orchestra.remote:timed out 2023-08-24T09:54:07.472 DEBUG:teuthology.orchestra.remote:timed out 2023-08-24T09:54:12.110 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2023-08-24T09:54:12.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-24T09:54:12.457 DEBUG:teuthology.orchestra.run.smithi135:> true 2023-08-24T09:54:12.781 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi135.front.sepia.ceph.com' 2023-08-24T09:54:12.782 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-359.el9.x86_64"... 2023-08-24T09:54:12.782 DEBUG:teuthology.orchestra.run.smithi135:> uname -r 2023-08-24T09:54:12.795 INFO:teuthology.orchestra.run.smithi135.stdout:5.14.0-359.el9.x86_64 2023-08-24T09:54:12.796 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-359.el9.x86_64 vs 5.14.0-359.el9.x86_64 2023-08-24T09:54:12.796 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-08-24T09:54:12.797 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-08-24T09:54:13.798 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2023-08-24T09:54:13.798 DEBUG:teuthology.orchestra.run.smithi135:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-08-24T09:54:13.867 INFO:teuthology.orchestra.run.smithi135.stdout:ttyS1 2023-08-24T09:54:13.901 DEBUG:teuthology.parallel:result is None 2023-08-24T09:54:16.473 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2023-08-24T09:54:16.474 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-24T09:54:17.058 DEBUG:teuthology.orchestra.run.smithi060:> true 2023-08-24T09:54:17.485 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2023-08-24T09:54:17.486 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-359.el9.x86_64"... 2023-08-24T09:54:17.486 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2023-08-24T09:54:17.501 INFO:teuthology.orchestra.run.smithi060.stdout:5.14.0-359.el9.x86_64 2023-08-24T09:54:17.501 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-359.el9.x86_64 vs 5.14.0-359.el9.x86_64 2023-08-24T09:54:17.502 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-08-24T09:54:17.502 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-08-24T09:54:18.503 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-08-24T09:54:18.504 DEBUG:teuthology.orchestra.run.smithi060:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-08-24T09:54:18.580 INFO:teuthology.orchestra.run.smithi060.stdout:ttyS1 2023-08-24T09:54:18.635 DEBUG:teuthology.parallel:result is None 2023-08-24T09:54:18.636 INFO:teuthology.run_tasks:Running task internal.base... 2023-08-24T09:54:18.661 INFO:teuthology.task.internal:Creating test directory... 2023-08-24T09:54:18.662 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-24T09:54:18.671 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-24T09:54:18.711 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-08-24T09:54:18.726 INFO:teuthology.run_tasks:Running task internal.archive... 2023-08-24T09:54:18.741 INFO:teuthology.task.internal:Creating archive directory... 2023-08-24T09:54:18.742 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-24T09:54:18.748 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-24T09:54:18.861 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-08-24T09:54:18.873 INFO:teuthology.task.internal:Enabling coredump saving... 2023-08-24T09:54:18.873 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-24T09:54:18.883 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-24T09:54:18.914 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-24T09:54:18.926 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-24T09:54:18.928 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-24T09:54:18.940 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-24T09:54:18.941 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-08-24T09:54:18.949 INFO:teuthology.task.internal:Configuring sudo... 2023-08-24T09:54:18.950 DEBUG:teuthology.orchestra.run.smithi060:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-24T09:54:18.977 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-24T09:54:19.021 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-08-24T09:54:19.042 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-08-24T09:54:19.043 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-24T09:54:19.058 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-24T09:54:19.090 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-24T09:54:19.163 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-24T09:54:19.237 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-24T09:54:19.281 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-24T09:54:19.338 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-08-24T09:54:19.338 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-24T09:54:19.412 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-24T09:54:19.457 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-24T09:54:19.516 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-24T09:54:19.597 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-24T09:54:19.644 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-24T09:54:19.645 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-24T09:54:19.714 DEBUG:teuthology.orchestra.run.smithi060:> sudo service rsyslog restart 2023-08-24T09:54:19.717 DEBUG:teuthology.orchestra.run.smithi135:> sudo service rsyslog restart 2023-08-24T09:54:19.768 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-24T09:54:19.822 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-24T09:54:20.126 INFO:teuthology.run_tasks:Running task internal.timer... 2023-08-24T09:54:20.134 INFO:teuthology.task.internal:Starting timer... 2023-08-24T09:54:20.135 INFO:teuthology.run_tasks:Running task pcp... 2023-08-24T09:54:20.190 INFO:teuthology.run_tasks:Running task selinux... 2023-08-24T09:54:20.203 DEBUG:teuthology.orchestra.run.smithi060:> sudo service auditd rotate 2023-08-24T09:54:20.257 INFO:teuthology.orchestra.run.smithi060.stdout:Rotating logs: 2023-08-24T09:54:20.260 DEBUG:teuthology.orchestra.run.smithi135:> sudo service auditd rotate 2023-08-24T09:54:20.308 INFO:teuthology.orchestra.run.smithi135.stdout:Rotating logs: 2023-08-24T09:54:20.310 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-08-24T09:54:20.311 DEBUG:teuthology.orchestra.run.smithi060:> /usr/sbin/getenforce 2023-08-24T09:54:20.329 INFO:teuthology.orchestra.run.smithi060.stdout:Permissive 2023-08-24T09:54:20.330 DEBUG:teuthology.orchestra.run.smithi135:> /usr/sbin/getenforce 2023-08-24T09:54:20.351 INFO:teuthology.orchestra.run.smithi135.stdout:Permissive 2023-08-24T09:54:20.352 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi060.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi135.front.sepia.ceph.com': 'permissive'} 2023-08-24T09:54:20.352 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-24T09:54:20.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-24T09:54:20.404 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-24T09:54:20.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-24T09:54:20.433 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-08-24T09:54:20.433 DEBUG:teuthology.orchestra.run.smithi060:> sudo /usr/sbin/setenforce permissive 2023-08-24T09:54:20.478 DEBUG:teuthology.orchestra.run.smithi135:> sudo /usr/sbin/setenforce permissive 2023-08-24T09:54:20.506 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-08-24T09:54:20.517 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-08-24T09:54:20.517 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-08-24T09:54:20.542 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-08-24T09:54:20.544 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi060.front.sepia.ceph.com,smithi135.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-08-24T10:01:08.116 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi060.front.sepia.ceph.com'), Remote(name='ubuntu@smithi135.front.sepia.ceph.com')] 2023-08-24T10:01:08.141 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2023-08-24T10:01:08.142 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-24T10:01:08.237 DEBUG:teuthology.orchestra.run.smithi060:> true 2023-08-24T10:01:08.308 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2023-08-24T10:01:08.309 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2023-08-24T10:01:08.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-24T10:01:08.393 DEBUG:teuthology.orchestra.run.smithi135:> true 2023-08-24T10:01:08.458 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi135.front.sepia.ceph.com' 2023-08-24T10:01:08.459 INFO:teuthology.run_tasks:Running task clock... 2023-08-24T10:01:08.473 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-08-24T10:01:08.474 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-24T10:01:08.475 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-24T10:01:08.478 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-24T10:01:08.479 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-24T10:01:08.510 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-24T10:01:08.530 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-24T10:01:08.531 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-24T10:01:08.548 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-24T10:01:08.566 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: ntpd: command not found 2023-08-24T10:01:08.577 INFO:teuthology.orchestra.run.smithi135.stderr:sudo: ntpd: command not found 2023-08-24T10:01:08.583 INFO:teuthology.orchestra.run.smithi060.stdout:506 Cannot talk to daemon 2023-08-24T10:01:08.591 INFO:teuthology.orchestra.run.smithi135.stdout:506 Cannot talk to daemon 2023-08-24T10:01:08.603 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-24T10:01:08.609 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-24T10:01:08.622 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-24T10:01:08.626 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-24T10:01:08.670 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2023-08-24T10:01:08.670 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2023-08-24T10:01:08.673 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-24T10:01:08.673 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2023-08-24T10:01:08.673 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-24T10:01:08.673 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-24T10:01:08.673 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-24T10:01:08.673 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-24T10:01:08.675 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-24T10:01:08.675 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2023-08-24T10:01:08.675 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-24T10:01:08.675 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-24T10:01:08.675 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-24T10:01:08.675 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-24T10:01:08.676 INFO:teuthology.run_tasks:Running task install... 2023-08-24T10:01:08.688 DEBUG:teuthology.task.install:project ceph 2023-08-24T10:01:08.689 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'crimson', 'sha1': '74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe'}} 2023-08-24T10:01:08.689 DEBUG:teuthology.task.install:config {'flavor': 'crimson', 'sha1': '74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe'} 2023-08-24T10:01:08.689 INFO:teuthology.task.install:Using flavor: crimson 2023-08-24T10:01:08.695 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-08-24T10:01:08.695 INFO:teuthology.task.install:extra packages: [] 2023-08-24T10:01:08.696 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe', 'tag': None, 'wait_for_package': False} 2023-08-24T10:01:08.696 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe 2023-08-24T10:01:08.698 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe', 'tag': None, 'wait_for_package': False} 2023-08-24T10:01:08.699 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe 2023-08-24T10:01:08.836 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-24.8-v2/74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/centos/9/flavors/crimson/ 2023-08-24T10:01:08.836 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5670.g74f8c0ae 2023-08-24T10:01:08.850 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-24.8-v2/74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/centos/9/flavors/crimson/ 2023-08-24T10:01:08.850 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5670.g74f8c0ae 2023-08-24T10:01:09.042 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-24.8-v2/74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-24.8-v2/74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-24.8-v2/74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-24T10:01:09.042 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-08-24T10:01:09.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-24T10:01:09.059 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-24.8-v2/74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-24.8-v2/74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-24.8-v2/74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-24T10:01:09.060 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-24T10:01:09.060 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-24T10:01:09.075 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-08-24T10:01:09.075 DEBUG:teuthology.orchestra.run.smithi060:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-24T10:01:09.089 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-08-24T10:01:09.090 DEBUG:teuthology.orchestra.run.smithi135:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-24T10:01:09.146 DEBUG:teuthology.orchestra.run.smithi060:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-24T10:01:09.159 DEBUG:teuthology.orchestra.run.smithi135:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-24T10:01:09.233 DEBUG:teuthology.orchestra.run.smithi060:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-24T10:01:09.239 DEBUG:teuthology.orchestra.run.smithi135:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-24T10:01:09.262 INFO:teuthology.orchestra.run.smithi060.stdout:check_obsoletes = 1 2023-08-24T10:01:09.264 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum clean all 2023-08-24T10:01:09.306 INFO:teuthology.orchestra.run.smithi135.stdout:check_obsoletes = 1 2023-08-24T10:01:09.308 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum clean all 2023-08-24T10:01:09.509 INFO:teuthology.orchestra.run.smithi060.stdout:72 files removed 2023-08-24T10:01:09.537 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-radosgw 2023-08-24T10:01:09.539 INFO:teuthology.orchestra.run.smithi135.stdout:72 files removed 2023-08-24T10:01:09.566 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-radosgw 2023-08-24T10:01:10.382 INFO:teuthology.orchestra.run.smithi135.stdout:ceph packages for x86_64 152 kB/s | 81 kB 00:00 2023-08-24T10:01:10.399 INFO:teuthology.orchestra.run.smithi060.stdout:ceph packages for x86_64 145 kB/s | 81 kB 00:00 2023-08-24T10:01:10.799 INFO:teuthology.orchestra.run.smithi135.stdout:ceph noarch packages 25 kB/s | 9.4 kB 00:00 2023-08-24T10:01:10.825 INFO:teuthology.orchestra.run.smithi060.stdout:ceph noarch packages 25 kB/s | 9.4 kB 00:00 2023-08-24T10:01:11.242 INFO:teuthology.orchestra.run.smithi135.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2023-08-24T10:01:11.267 INFO:teuthology.orchestra.run.smithi060.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2023-08-24T10:01:11.919 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS-9 - AppStream 9.6 MB/s | 6.2 MB 00:00 2023-08-24T10:01:12.011 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS-9 - AppStream 8.7 MB/s | 6.2 MB 00:00 2023-08-24T10:01:13.428 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS-9 - BaseOS 5.4 MB/s | 2.0 MB 00:00 2023-08-24T10:01:13.553 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS-9 - BaseOS 3.2 MB/s | 2.0 MB 00:00 2023-08-24T10:01:14.194 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 7.8 MB 00:00 2023-08-24T10:01:15.069 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 7.8 MB 00:01 2023-08-24T10:01:23.169 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - AppStream 2.5 MB/s | 18 MB 00:07 2023-08-24T10:01:23.254 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 2.2 MB/s | 18 MB 00:08 2023-08-24T10:01:26.812 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - CRB 13 MB/s | 5.7 MB 00:00 2023-08-24T10:01:28.121 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 14 kB 00:00 2023-08-24T10:01:28.521 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 5.7 MB 00:01 2023-08-24T10:01:29.706 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 12 MB/s | 18 MB 00:01 2023-08-24T10:01:30.723 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 14 kB 00:01 2023-08-24T10:01:34.184 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 5.5 MB/s | 18 MB 00:03 2023-08-24T10:01:35.829 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:01:35.840 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:01:35.840 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-08-24T10:01:35.841 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:01:35.841 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-08-24T10:01:35.841 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 8.2 M 2023-08-24T10:01:35.842 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrading: 2023-08-24T10:01:35.842 INFO:teuthology.orchestra.run.smithi135.stdout: librados2 x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 3.1 M 2023-08-24T10:01:35.842 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1 x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 3.0 M 2023-08-24T10:01:35.842 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-08-24T10:01:35.843 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options 2023-08-24T10:01:35.843 INFO:teuthology.orchestra.run.smithi135.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-08-24T10:01:35.843 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 6.6 M 2023-08-24T10:01:35.843 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 17 M 2023-08-24T10:01:35.844 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 24 k 2023-08-24T10:01:35.844 INFO:teuthology.orchestra.run.smithi135.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-08-24T10:01:35.844 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-08-24T10:01:35.845 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-08-24T10:01:35.845 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-08-24T10:01:35.846 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2 x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 680 k 2023-08-24T10:01:35.846 INFO:teuthology.orchestra.run.smithi135.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-08-24T10:01:35.846 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-08-24T10:01:35.846 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-08-24T10:01:35.847 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1 x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 469 k 2023-08-24T10:01:35.847 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-08-24T10:01:35.848 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2 x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 4.4 M 2023-08-24T10:01:35.848 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-08-24T10:01:35.848 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-08-24T10:01:35.849 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-08-24T10:01:35.849 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse 2023-08-24T10:01:35.849 INFO:teuthology.orchestra.run.smithi135.stdout: x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 45 k 2023-08-24T10:01:35.849 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 127 k 2023-08-24T10:01:35.850 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 161 k 2023-08-24T10:01:35.850 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-08-24T10:01:35.850 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 320 k 2023-08-24T10:01:35.850 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 297 k 2023-08-24T10:01:35.851 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 99 k 2023-08-24T10:01:35.851 INFO:teuthology.orchestra.run.smithi135.stdout: re2 x86_64 1:20211101-2.el9 epel 192 k 2023-08-24T10:01:35.851 INFO:teuthology.orchestra.run.smithi135.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-08-24T10:01:35.851 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:01:35.852 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-08-24T10:01:35.852 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:01:35.852 INFO:teuthology.orchestra.run.smithi135.stdout:Install 28 Packages 2023-08-24T10:01:35.852 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrade 2 Packages 2023-08-24T10:01:35.853 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:01:35.853 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 53 M 2023-08-24T10:01:35.854 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-08-24T10:01:38.098 INFO:teuthology.orchestra.run.smithi135.stdout:(1/30): ceph-base-18.0.0-5670.g74f8c0ae.el9.x86 5.4 MB/s | 6.6 MB 00:01 2023-08-24T10:01:38.232 INFO:teuthology.orchestra.run.smithi135.stdout:(2/30): ceph-radosgw-18.0.0-5670.g74f8c0ae.el9. 6.1 MB/s | 8.2 MB 00:01 2023-08-24T10:01:38.257 INFO:teuthology.orchestra.run.smithi135.stdout:(3/30): ceph-selinux-18.0.0-5670.g74f8c0ae.el9. 152 kB/s | 24 kB 00:00 2023-08-24T10:01:38.365 INFO:teuthology.orchestra.run.smithi135.stdout:(4/30): libcephfs2-18.0.0-5670.g74f8c0ae.el9.x8 5.0 MB/s | 680 kB 00:00 2023-08-24T10:01:38.407 INFO:teuthology.orchestra.run.smithi135.stdout:(5/30): libradosstriper1-18.0.0-5670.g74f8c0ae. 3.0 MB/s | 469 kB 00:00 2023-08-24T10:01:38.491 INFO:teuthology.orchestra.run.smithi135.stdout:(6/30): python3-ceph-argparse-18.0.0-5670.g74f8 543 kB/s | 45 kB 00:00 2023-08-24T10:01:38.608 INFO:teuthology.orchestra.run.smithi135.stdout:(7/30): python3-ceph-common-18.0.0-5670.g74f8c0 1.1 MB/s | 127 kB 00:00 2023-08-24T10:01:38.883 INFO:teuthology.orchestra.run.smithi135.stdout:(8/30): ceph-common-18.0.0-5670.g74f8c0ae.el9.x 8.6 MB/s | 17 MB 00:01 2023-08-24T10:01:38.917 INFO:teuthology.orchestra.run.smithi135.stdout:(9/30): python3-cephfs-18.0.0-5670.g74f8c0ae.el 523 kB/s | 161 kB 00:00 2023-08-24T10:01:39.017 INFO:teuthology.orchestra.run.smithi135.stdout:(10/30): librgw2-18.0.0-5670.g74f8c0ae.el9.x86_ 6.7 MB/s | 4.4 MB 00:00 2023-08-24T10:01:39.051 INFO:teuthology.orchestra.run.smithi135.stdout:(11/30): python3-rados-18.0.0-5670.g74f8c0ae.el 1.9 MB/s | 320 kB 00:00 2023-08-24T10:01:39.093 INFO:teuthology.orchestra.run.smithi135.stdout:(12/30): python3-rbd-18.0.0-5670.g74f8c0ae.el9. 1.7 MB/s | 297 kB 00:00 2023-08-24T10:01:39.151 INFO:teuthology.orchestra.run.smithi135.stdout:(13/30): python3-rgw-18.0.0-5670.g74f8c0ae.el9. 743 kB/s | 99 kB 00:00 2023-08-24T10:01:39.235 INFO:teuthology.orchestra.run.smithi135.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 576 kB/s | 106 kB 00:00 2023-08-24T10:01:39.260 INFO:teuthology.orchestra.run.smithi135.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 529 kB/s | 44 kB 00:00 2023-08-24T10:01:39.319 INFO:teuthology.orchestra.run.smithi135.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2023-08-24T10:01:39.361 INFO:teuthology.orchestra.run.smithi135.stdout:(17/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 655 kB/s | 159 kB 00:00 2023-08-24T10:01:39.403 INFO:teuthology.orchestra.run.smithi135.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 282 kB 00:00 2023-08-24T10:01:39.420 INFO:teuthology.orchestra.run.smithi135.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 412 kB/s | 41 kB 00:00 2023-08-24T10:01:39.637 INFO:teuthology.orchestra.run.smithi135.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 474 kB/s | 111 kB 00:00 2023-08-24T10:01:39.670 INFO:teuthology.orchestra.run.smithi135.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.1 MB/s | 309 kB 00:00 2023-08-24T10:01:39.695 INFO:teuthology.orchestra.run.smithi135.stdout:(22/30): libarrow-doc-9.0.0-9.el9.noarch.rpm 416 kB/s | 24 kB 00:00 2023-08-24T10:01:39.721 INFO:teuthology.orchestra.run.smithi135.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2023-08-24T10:01:39.764 INFO:teuthology.orchestra.run.smithi135.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2023-08-24T10:01:39.838 INFO:teuthology.orchestra.run.smithi135.stdout:(25/30): parquet-libs-9.0.0-9.el9.x86_64.rpm 7.0 MB/s | 837 kB 00:00 2023-08-24T10:01:39.872 INFO:teuthology.orchestra.run.smithi135.stdout:(26/30): re2-20211101-2.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2023-08-24T10:01:39.964 INFO:teuthology.orchestra.run.smithi135.stdout:(27/30): libarrow-9.0.0-9.el9.x86_64.rpm 8.1 MB/s | 4.4 MB 00:00 2023-08-24T10:01:40.014 INFO:teuthology.orchestra.run.smithi135.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2023-08-24T10:01:40.356 INFO:teuthology.orchestra.run.smithi135.stdout:(29/30): librados2-18.0.0-5670.g74f8c0ae.el9.x8 6.3 MB/s | 3.1 MB 00:00 2023-08-24T10:01:40.539 INFO:teuthology.orchestra.run.smithi135.stdout:(30/30): librbd1-18.0.0-5670.g74f8c0ae.el9.x86_ 5.3 MB/s | 3.0 MB 00:00 2023-08-24T10:01:40.546 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:01:40.546 INFO:teuthology.orchestra.run.smithi135.stdout:Total 11 MB/s | 53 MB 00:04 2023-08-24T10:01:40.547 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-08-24T10:01:40.579 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-08-24T10:01:40.579 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-08-24T10:01:40.625 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:01:40.639 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:01:40.639 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2023-08-24T10:01:40.639 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:01:40.639 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-08-24T10:01:40.639 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 8.2 M 2023-08-24T10:01:40.640 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrading: 2023-08-24T10:01:40.640 INFO:teuthology.orchestra.run.smithi060.stdout: librados2 x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 3.1 M 2023-08-24T10:01:40.640 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1 x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 3.0 M 2023-08-24T10:01:40.640 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2023-08-24T10:01:40.640 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options 2023-08-24T10:01:40.640 INFO:teuthology.orchestra.run.smithi060.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-08-24T10:01:40.640 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 6.6 M 2023-08-24T10:01:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 17 M 2023-08-24T10:01:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 24 k 2023-08-24T10:01:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-08-24T10:01:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-08-24T10:01:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-08-24T10:01:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-08-24T10:01:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2 x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 680 k 2023-08-24T10:01:40.641 INFO:teuthology.orchestra.run.smithi060.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-08-24T10:01:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-08-24T10:01:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-08-24T10:01:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1 x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 469 k 2023-08-24T10:01:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-08-24T10:01:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2 x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 4.4 M 2023-08-24T10:01:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-08-24T10:01:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-08-24T10:01:40.642 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-08-24T10:01:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse 2023-08-24T10:01:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 45 k 2023-08-24T10:01:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 127 k 2023-08-24T10:01:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 161 k 2023-08-24T10:01:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-08-24T10:01:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 320 k 2023-08-24T10:01:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 297 k 2023-08-24T10:01:40.643 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 99 k 2023-08-24T10:01:40.644 INFO:teuthology.orchestra.run.smithi060.stdout: re2 x86_64 1:20211101-2.el9 epel 192 k 2023-08-24T10:01:40.644 INFO:teuthology.orchestra.run.smithi060.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-08-24T10:01:40.644 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:01:40.644 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-08-24T10:01:40.644 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:01:40.644 INFO:teuthology.orchestra.run.smithi060.stdout:Install 28 Packages 2023-08-24T10:01:40.644 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrade 2 Packages 2023-08-24T10:01:40.645 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:01:40.645 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 53 M 2023-08-24T10:01:40.645 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-08-24T10:01:41.027 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-08-24T10:01:41.028 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-08-24T10:01:41.607 INFO:teuthology.orchestra.run.smithi060.stdout:(1/30): ceph-radosgw-18.0.0-5670.g74f8c0ae.el9. 10 MB/s | 8.2 MB 00:00 2023-08-24T10:01:41.649 INFO:teuthology.orchestra.run.smithi060.stdout:(2/30): ceph-selinux-18.0.0-5670.g74f8c0ae.el9. 584 kB/s | 24 kB 00:00 2023-08-24T10:01:41.743 INFO:teuthology.orchestra.run.smithi060.stdout:(3/30): libcephfs2-18.0.0-5670.g74f8c0ae.el9.x8 7.1 MB/s | 680 kB 00:00 2023-08-24T10:01:41.894 INFO:teuthology.orchestra.run.smithi060.stdout:(4/30): ceph-base-18.0.0-5670.g74f8c0ae.el9.x86 6.1 MB/s | 6.6 MB 00:01 2023-08-24T10:01:41.935 INFO:teuthology.orchestra.run.smithi060.stdout:(5/30): libradosstriper1-18.0.0-5670.g74f8c0ae. 2.4 MB/s | 469 kB 00:00 2023-08-24T10:01:41.986 INFO:teuthology.orchestra.run.smithi060.stdout:(6/30): python3-ceph-argparse-18.0.0-5670.g74f8 906 kB/s | 45 kB 00:00 2023-08-24T10:01:42.069 INFO:teuthology.orchestra.run.smithi060.stdout:(7/30): python3-ceph-common-18.0.0-5670.g74f8c0 1.5 MB/s | 127 kB 00:00 2023-08-24T10:01:42.161 INFO:teuthology.orchestra.run.smithi060.stdout:(8/30): python3-cephfs-18.0.0-5670.g74f8c0ae.el 1.7 MB/s | 161 kB 00:00 2023-08-24T10:01:42.253 INFO:teuthology.orchestra.run.smithi060.stdout:(9/30): librgw2-18.0.0-5670.g74f8c0ae.el9.x86_6 12 MB/s | 4.4 MB 00:00 2023-08-24T10:01:42.287 INFO:teuthology.orchestra.run.smithi060.stdout:(10/30): python3-rados-18.0.0-5670.g74f8c0ae.el 2.5 MB/s | 320 kB 00:00 2023-08-24T10:01:42.337 INFO:teuthology.orchestra.run.smithi060.stdout:(11/30): python3-rbd-18.0.0-5670.g74f8c0ae.el9. 3.5 MB/s | 297 kB 00:00 2023-08-24T10:01:42.367 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-08-24T10:01:42.371 INFO:teuthology.orchestra.run.smithi060.stdout:(12/30): python3-rgw-18.0.0-5670.g74f8c0ae.el9. 1.2 MB/s | 99 kB 00:00 2023-08-24T10:01:42.522 INFO:teuthology.orchestra.run.smithi060.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 576 kB/s | 106 kB 00:00 2023-08-24T10:01:42.555 INFO:teuthology.orchestra.run.smithi060.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 159 kB 00:00 2023-08-24T10:01:42.580 INFO:teuthology.orchestra.run.smithi060.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 755 kB/s | 44 kB 00:00 2023-08-24T10:01:42.639 INFO:teuthology.orchestra.run.smithi060.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2023-08-24T10:01:42.672 INFO:teuthology.orchestra.run.smithi060.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.0 MB/s | 282 kB 00:00 2023-08-24T10:01:42.698 INFO:teuthology.orchestra.run.smithi060.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 710 kB/s | 41 kB 00:00 2023-08-24T10:01:42.811 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2023-08-24T10:01:42.875 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librados2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2/32 2023-08-24T10:01:42.898 INFO:teuthology.orchestra.run.smithi060.stdout:(19/30): fmt-8.1.1-5.el9.x86_64.rpm 492 kB/s | 111 kB 00:00 2023-08-24T10:01:42.966 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2/32 2023-08-24T10:01:43.071 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rados-2:18.0.0-5670.g74f8c0ae.el9.x86_64 3/32 2023-08-24T10:01:43.099 INFO:teuthology.orchestra.run.smithi060.stdout:(20/30): ceph-common-18.0.0-5670.g74f8c0ae.el9. 7.5 MB/s | 17 MB 00:02 2023-08-24T10:01:43.110 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 4/32 2023-08-24T10:01:43.132 INFO:teuthology.orchestra.run.smithi060.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 711 kB/s | 309 kB 00:00 2023-08-24T10:01:43.191 INFO:teuthology.orchestra.run.smithi060.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 833 kB/s | 49 kB 00:00 2023-08-24T10:01:43.192 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephfs2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 4/32 2023-08-24T10:01:43.224 INFO:teuthology.orchestra.run.smithi060.stdout:(23/30): libarrow-doc-9.0.0-9.el9.noarch.rpm 195 kB/s | 24 kB 00:00 2023-08-24T10:01:43.231 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/32 2023-08-24T10:01:43.258 INFO:teuthology.orchestra.run.smithi060.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2023-08-24T10:01:43.265 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 6/32 2023-08-24T10:01:43.302 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : liboath-2.6.7-2.el9.x86_64 7/32 2023-08-24T10:01:43.341 INFO:teuthology.orchestra.run.smithi060.stdout:(25/30): re2-20211101-2.el9.x86_64.rpm 2.3 MB/s | 192 kB 00:00 2023-08-24T10:01:43.354 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-argparse-2:18.0.0-5670.g74f8c0ae.el9. 8/32 2023-08-24T10:01:43.437 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cephfs-2:18.0.0-5670.g74f8c0ae.el9.x86_64 9/32 2023-08-24T10:01:43.458 INFO:teuthology.orchestra.run.smithi060.stdout:(26/30): parquet-libs-9.0.0-9.el9.x86_64.rpm 3.5 MB/s | 837 kB 00:00 2023-08-24T10:01:43.472 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libradosstriper1-2:18.0.0-5670.g74f8c0ae.el9.x86_6 10/32 2023-08-24T10:01:43.543 INFO:teuthology.orchestra.run.smithi060.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2023-08-24T10:01:43.568 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5670.g74f8c0ae.el9.x86_6 10/32 2023-08-24T10:01:43.620 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : thrift-0.15.0-2.el9.x86_64 11/32 2023-08-24T10:01:43.744 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : re2-1:20211101-2.el9.x86_64 12/32 2023-08-24T10:01:43.786 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 13/32 2023-08-24T10:01:43.801 INFO:teuthology.orchestra.run.smithi060.stdout:(28/30): libarrow-9.0.0-9.el9.x86_64.rpm 4.9 MB/s | 4.4 MB 00:00 2023-08-24T10:01:43.829 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2023-08-24T10:01:43.869 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2023-08-24T10:01:43.876 INFO:teuthology.orchestra.run.smithi060.stdout:(29/30): librados2-18.0.0-5670.g74f8c0ae.el9.x8 7.4 MB/s | 3.1 MB 00:00 2023-08-24T10:01:43.905 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 16/32 2023-08-24T10:01:43.944 INFO:teuthology.orchestra.run.smithi060.stdout:(30/30): librbd1-18.0.0-5670.g74f8c0ae.el9.x86_ 7.6 MB/s | 3.0 MB 00:00 2023-08-24T10:01:43.947 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:01:43.947 INFO:teuthology.orchestra.run.smithi060.stdout:Total 16 MB/s | 53 MB 00:03 2023-08-24T10:01:43.948 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-08-24T10:01:43.957 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2023-08-24T10:01:43.989 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-08-24T10:01:43.990 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-08-24T10:01:44.006 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-08-24T10:01:44.352 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2023-08-24T10:01:44.386 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librbd1-2:18.0.0-5670.g74f8c0ae.el9.x86_64 20/32 2023-08-24T10:01:44.451 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-08-24T10:01:44.452 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-08-24T10:01:44.467 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:18.0.0-5670.g74f8c0ae.el9.x86_64 20/32 2023-08-24T10:01:44.509 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rbd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 21/32 2023-08-24T10:01:44.566 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2023-08-24T10:01:45.000 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 23/32 2023-08-24T10:01:45.041 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librgw2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 24/32 2023-08-24T10:01:45.111 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librgw2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 24/32 2023-08-24T10:01:45.188 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rgw-2:18.0.0-5670.g74f8c0ae.el9.x86_64 25/32 2023-08-24T10:01:45.226 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-common-2:18.0.0-5670.g74f8c0ae.el9.x8 26/32 2023-08-24T10:01:46.018 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-08-24T10:01:46.444 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2023-08-24T10:01:46.529 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librados2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2/32 2023-08-24T10:01:46.638 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2/32 2023-08-24T10:01:46.742 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rados-2:18.0.0-5670.g74f8c0ae.el9.x86_64 3/32 2023-08-24T10:01:46.780 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 4/32 2023-08-24T10:01:46.859 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephfs2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 4/32 2023-08-24T10:01:46.909 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/32 2023-08-24T10:01:46.951 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 6/32 2023-08-24T10:01:46.998 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : liboath-2.6.7-2.el9.x86_64 7/32 2023-08-24T10:01:47.092 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-argparse-2:18.0.0-5670.g74f8c0ae.el9. 8/32 2023-08-24T10:01:47.183 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cephfs-2:18.0.0-5670.g74f8c0ae.el9.x86_64 9/32 2023-08-24T10:01:47.194 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:18.0.0-5670.g74f8c0ae.el9.x86_64 27/32 2023-08-24T10:01:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libradosstriper1-2:18.0.0-5670.g74f8c0ae.el9.x86_6 10/32 2023-08-24T10:01:47.239 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-common-2:18.0.0-5670.g74f8c0ae.el9.x86_64 27/32 2023-08-24T10:01:47.320 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5670.g74f8c0ae.el9.x86_6 10/32 2023-08-24T10:01:47.366 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : thrift-0.15.0-2.el9.x86_64 11/32 2023-08-24T10:01:47.484 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : re2-1:20211101-2.el9.x86_64 12/32 2023-08-24T10:01:47.526 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 13/32 2023-08-24T10:01:47.582 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2023-08-24T10:01:47.641 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2023-08-24T10:01:47.684 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 16/32 2023-08-24T10:01:47.736 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2023-08-24T10:01:47.786 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-08-24T10:01:47.986 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:18.0.0-5670.g74f8c0ae.el9.x86_64 27/32 2023-08-24T10:01:48.109 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-base-2:18.0.0-5670.g74f8c0ae.el9.x86_64 28/32 2023-08-24T10:01:48.122 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2023-08-24T10:01:48.157 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librbd1-2:18.0.0-5670.g74f8c0ae.el9.x86_64 20/32 2023-08-24T10:01:48.183 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-base-2:18.0.0-5670.g74f8c0ae.el9.x86_64 28/32 2023-08-24T10:01:48.184 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-08-24T10:01:48.184 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-08-24T10:01:48.184 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:01:48.228 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-selinux-2:18.0.0-5670.g74f8c0ae.el9.x86_64 29/32 2023-08-24T10:01:48.233 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:18.0.0-5670.g74f8c0ae.el9.x86_64 20/32 2023-08-24T10:01:48.269 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rbd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 21/32 2023-08-24T10:01:48.315 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2023-08-24T10:01:48.776 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 23/32 2023-08-24T10:01:48.819 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librgw2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 24/32 2023-08-24T10:01:48.889 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librgw2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 24/32 2023-08-24T10:01:48.988 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rgw-2:18.0.0-5670.g74f8c0ae.el9.x86_64 25/32 2023-08-24T10:01:49.029 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-common-2:18.0.0-5670.g74f8c0ae.el9.x8 26/32 2023-08-24T10:01:51.175 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:18.0.0-5670.g74f8c0ae.el9.x86_64 27/32 2023-08-24T10:01:51.225 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-common-2:18.0.0-5670.g74f8c0ae.el9.x86_64 27/32 2023-08-24T10:01:51.981 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:18.0.0-5670.g74f8c0ae.el9.x86_64 27/32 2023-08-24T10:01:52.037 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-base-2:18.0.0-5670.g74f8c0ae.el9.x86_64 28/32 2023-08-24T10:01:52.125 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-base-2:18.0.0-5670.g74f8c0ae.el9.x86_64 28/32 2023-08-24T10:01:52.125 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-08-24T10:01:52.125 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-08-24T10:01:52.126 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:01:52.163 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-selinux-2:18.0.0-5670.g74f8c0ae.el9.x86_64 29/32 2023-08-24T10:01:58.500 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5670.g74f8c0ae.el9.x86_64 29/32 2023-08-24T10:01:58.500 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /sys 2023-08-24T10:01:58.501 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /proc 2023-08-24T10:01:58.501 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /mnt 2023-08-24T10:01:58.501 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /var/tmp 2023-08-24T10:01:58.502 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /home 2023-08-24T10:01:58.502 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /root 2023-08-24T10:01:58.502 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /tmp 2023-08-24T10:01:58.503 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:01:59.316 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-radosgw-2:18.0.0-5670.g74f8c0ae.el9.x86_64 30/32 2023-08-24T10:01:59.341 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5670.g74f8c0ae.el9.x86_64 30/32 2023-08-24T10:01:59.341 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-24T10:01:59.341 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-08-24T10:01:59.342 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-08-24T10:01:59.342 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-08-24T10:01:59.342 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:01:59.343 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-08-24T10:01:59.435 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-08-24T10:01:59.436 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2023-08-24T10:02:00.835 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2023-08-24T10:02:00.836 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-base-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/32 2023-08-24T10:02:00.836 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-common-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2/32 2023-08-24T10:02:00.836 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-radosgw-2:18.0.0-5670.g74f8c0ae.el9.x86_64 3/32 2023-08-24T10:02:00.836 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-selinux-2:18.0.0-5670.g74f8c0ae.el9.x86_64 4/32 2023-08-24T10:02:00.836 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 5/32 2023-08-24T10:02:00.836 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libradosstriper1-2:18.0.0-5670.g74f8c0ae.el9.x86_6 6/32 2023-08-24T10:02:00.837 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librgw2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 7/32 2023-08-24T10:02:00.837 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5670.g74f8c0ae.el9. 8/32 2023-08-24T10:02:00.837 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-common-2:18.0.0-5670.g74f8c0ae.el9.x8 9/32 2023-08-24T10:02:00.837 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cephfs-2:18.0.0-5670.g74f8c0ae.el9.x86_64 10/32 2023-08-24T10:02:00.837 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rados-2:18.0.0-5670.g74f8c0ae.el9.x86_64 11/32 2023-08-24T10:02:00.837 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rbd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 12/32 2023-08-24T10:02:00.837 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rgw-2:18.0.0-5670.g74f8c0ae.el9.x86_64 13/32 2023-08-24T10:02:00.837 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2023-08-24T10:02:00.838 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2023-08-24T10:02:00.838 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2023-08-24T10:02:00.838 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2023-08-24T10:02:00.838 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2023-08-24T10:02:00.838 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2023-08-24T10:02:00.838 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2023-08-24T10:02:00.838 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2023-08-24T10:02:00.839 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 22/32 2023-08-24T10:02:00.839 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 23/32 2023-08-24T10:02:00.840 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2023-08-24T10:02:00.840 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2023-08-24T10:02:00.840 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 26/32 2023-08-24T10:02:00.840 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : re2-1:20211101-2.el9.x86_64 27/32 2023-08-24T10:02:00.841 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2023-08-24T10:02:00.841 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 29/32 2023-08-24T10:02:00.841 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2023-08-24T10:02:00.841 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:18.0.0-5670.g74f8c0ae.el9.x86_64 31/32 2023-08-24T10:02:01.564 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2023-08-24T10:02:01.565 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:02:01.565 INFO:teuthology.orchestra.run.smithi135.stdout:Upgraded: 2023-08-24T10:02:01.565 INFO:teuthology.orchestra.run.smithi135.stdout: librados2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:01.565 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:01.565 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-08-24T10:02:01.566 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-08-24T10:02:01.566 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:01.566 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:01.566 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:01.566 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:01.566 INFO:teuthology.orchestra.run.smithi135.stdout: fmt-8.1.1-5.el9.x86_64 2023-08-24T10:02:01.566 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-08-24T10:02:01.567 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-9.0.0-9.el9.x86_64 2023-08-24T10:02:01.567 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-08-24T10:02:01.567 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:01.567 INFO:teuthology.orchestra.run.smithi135.stdout: liboath-2.6.7-2.el9.x86_64 2023-08-24T10:02:01.567 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-08-24T10:02:01.567 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-08-24T10:02:01.567 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:01.567 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-08-24T10:02:01.568 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:01.568 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind-1.6.2-1.el9.x86_64 2023-08-24T10:02:01.568 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-08-24T10:02:01.568 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-08-24T10:02:01.568 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:01.568 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:01.568 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:01.568 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-08-24T10:02:01.569 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:01.569 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:01.569 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:01.569 INFO:teuthology.orchestra.run.smithi135.stdout: re2-1:20211101-2.el9.x86_64 2023-08-24T10:02:01.569 INFO:teuthology.orchestra.run.smithi135.stdout: thrift-0.15.0-2.el9.x86_64 2023-08-24T10:02:01.569 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:02:01.569 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:02:01.975 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-test 2023-08-24T10:02:02.362 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5670.g74f8c0ae.el9.x86_64 29/32 2023-08-24T10:02:02.362 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /sys 2023-08-24T10:02:02.362 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /proc 2023-08-24T10:02:02.363 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /mnt 2023-08-24T10:02:02.363 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /var/tmp 2023-08-24T10:02:02.363 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /home 2023-08-24T10:02:02.363 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /root 2023-08-24T10:02:02.363 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /tmp 2023-08-24T10:02:02.363 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:02:02.493 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:00:33 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:02:02.626 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:02:02.627 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:02:02.627 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-08-24T10:02:02.628 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:02:02.628 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-08-24T10:02:02.628 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 116 M 2023-08-24T10:02:02.628 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-08-24T10:02:02.628 INFO:teuthology.orchestra.run.smithi135.stdout: cryptopp x86_64 8.6.0-1.el9 epel 1.3 M 2023-08-24T10:02:02.628 INFO:teuthology.orchestra.run.smithi135.stdout: jq x86_64 1.6-15.el9 CentOS-AppStream 186 k 2023-08-24T10:02:02.628 INFO:teuthology.orchestra.run.smithi135.stdout: libasan x86_64 11.4.1-2.1.el9 CentOS-AppStream 420 k 2023-08-24T10:02:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 161 k 2023-08-24T10:02:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: libubsan x86_64 11.4.1-2.1.el9 CentOS-AppStream 189 k 2023-08-24T10:02:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-08-24T10:02:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-08-24T10:02:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-08-24T10:02:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2023-08-24T10:02:02.629 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:02:02.630 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-08-24T10:02:02.630 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:02:02.630 INFO:teuthology.orchestra.run.smithi135.stdout:Install 10 Packages 2023-08-24T10:02:02.630 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:02:02.631 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 119 M 2023-08-24T10:02:02.631 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 1.1 G 2023-08-24T10:02:02.631 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-08-24T10:02:02.832 INFO:teuthology.orchestra.run.smithi135.stdout:(1/10): jq-1.6-15.el9.x86_64.rpm 1.2 MB/s | 186 kB 00:00 2023-08-24T10:02:02.899 INFO:teuthology.orchestra.run.smithi135.stdout:(2/10): libasan-11.4.1-2.1.el9.x86_64.rpm 6.2 MB/s | 420 kB 00:00 2023-08-24T10:02:02.949 INFO:teuthology.orchestra.run.smithi135.stdout:(3/10): libubsan-11.4.1-2.1.el9.x86_64.rpm 3.7 MB/s | 189 kB 00:00 2023-08-24T10:02:03.016 INFO:teuthology.orchestra.run.smithi135.stdout:(4/10): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2023-08-24T10:02:03.066 INFO:teuthology.orchestra.run.smithi135.stdout:(5/10): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2023-08-24T10:02:03.100 INFO:teuthology.orchestra.run.smithi135.stdout:(6/10): libcephsqlite-18.0.0-5670.g74f8c0ae.el9 382 kB/s | 161 kB 00:00 2023-08-24T10:02:03.134 INFO:teuthology.orchestra.run.smithi135.stdout:(7/10): xmlstarlet-1.6.1-20.el9.x86_64.rpm 933 kB/s | 63 kB 00:00 2023-08-24T10:02:03.195 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-radosgw-2:18.0.0-5670.g74f8c0ae.el9.x86_64 30/32 2023-08-24T10:02:03.221 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5670.g74f8c0ae.el9.x86_64 30/32 2023-08-24T10:02:03.221 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-24T10:02:03.221 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-08-24T10:02:03.221 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-08-24T10:02:03.222 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-08-24T10:02:03.222 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:02:03.223 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-08-24T10:02:03.306 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-08-24T10:02:03.306 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2023-08-24T10:02:03.767 INFO:teuthology.orchestra.run.smithi135.stdout:(8/10): yaml-cpp-0.6.3-5.el9.x86_64.rpm 192 kB/s | 122 kB 00:00 2023-08-24T10:02:04.167 INFO:teuthology.orchestra.run.smithi135.stdout:(9/10): cryptopp-8.6.0-1.el9.x86_64.rpm 1.2 MB/s | 1.3 MB 00:01 2023-08-24T10:02:04.624 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2023-08-24T10:02:04.625 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-base-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/32 2023-08-24T10:02:04.625 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-common-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2/32 2023-08-24T10:02:04.625 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-radosgw-2:18.0.0-5670.g74f8c0ae.el9.x86_64 3/32 2023-08-24T10:02:04.625 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-selinux-2:18.0.0-5670.g74f8c0ae.el9.x86_64 4/32 2023-08-24T10:02:04.625 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 5/32 2023-08-24T10:02:04.625 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libradosstriper1-2:18.0.0-5670.g74f8c0ae.el9.x86_6 6/32 2023-08-24T10:02:04.625 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librgw2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 7/32 2023-08-24T10:02:04.626 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5670.g74f8c0ae.el9. 8/32 2023-08-24T10:02:04.626 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-common-2:18.0.0-5670.g74f8c0ae.el9.x8 9/32 2023-08-24T10:02:04.626 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cephfs-2:18.0.0-5670.g74f8c0ae.el9.x86_64 10/32 2023-08-24T10:02:04.626 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rados-2:18.0.0-5670.g74f8c0ae.el9.x86_64 11/32 2023-08-24T10:02:04.626 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rbd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 12/32 2023-08-24T10:02:04.626 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rgw-2:18.0.0-5670.g74f8c0ae.el9.x86_64 13/32 2023-08-24T10:02:04.626 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2023-08-24T10:02:04.627 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2023-08-24T10:02:04.627 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2023-08-24T10:02:04.627 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2023-08-24T10:02:04.627 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2023-08-24T10:02:04.627 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2023-08-24T10:02:04.627 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2023-08-24T10:02:04.627 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2023-08-24T10:02:04.628 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 22/32 2023-08-24T10:02:04.628 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 23/32 2023-08-24T10:02:04.628 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2023-08-24T10:02:04.628 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2023-08-24T10:02:04.629 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 26/32 2023-08-24T10:02:04.629 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : re2-1:20211101-2.el9.x86_64 27/32 2023-08-24T10:02:04.630 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2023-08-24T10:02:04.630 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 29/32 2023-08-24T10:02:04.630 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2023-08-24T10:02:04.630 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:18.0.0-5670.g74f8c0ae.el9.x86_64 31/32 2023-08-24T10:02:05.393 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2023-08-24T10:02:05.393 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:02:05.393 INFO:teuthology.orchestra.run.smithi060.stdout:Upgraded: 2023-08-24T10:02:05.393 INFO:teuthology.orchestra.run.smithi060.stdout: librados2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:05.393 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:05.394 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-08-24T10:02:05.394 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-08-24T10:02:05.394 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:05.394 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:05.394 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:05.394 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:05.395 INFO:teuthology.orchestra.run.smithi060.stdout: fmt-8.1.1-5.el9.x86_64 2023-08-24T10:02:05.395 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-08-24T10:02:05.395 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-9.0.0-9.el9.x86_64 2023-08-24T10:02:05.396 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-08-24T10:02:05.396 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:05.396 INFO:teuthology.orchestra.run.smithi060.stdout: liboath-2.6.7-2.el9.x86_64 2023-08-24T10:02:05.396 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-08-24T10:02:05.397 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-08-24T10:02:05.397 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:05.397 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-08-24T10:02:05.397 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:05.397 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind-1.6.2-1.el9.x86_64 2023-08-24T10:02:05.398 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-08-24T10:02:05.398 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-08-24T10:02:05.398 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:05.399 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:05.399 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:05.399 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-08-24T10:02:05.400 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: re2-1:20211101-2.el9.x86_64 2023-08-24T10:02:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: thrift-0.15.0-2.el9.x86_64 2023-08-24T10:02:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:02:05.401 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:02:05.824 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-test 2023-08-24T10:02:06.360 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:00:32 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:02:06.497 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:02:06.499 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:02:06.499 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2023-08-24T10:02:06.499 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:02:06.499 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-08-24T10:02:06.500 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 116 M 2023-08-24T10:02:06.500 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2023-08-24T10:02:06.500 INFO:teuthology.orchestra.run.smithi060.stdout: cryptopp x86_64 8.6.0-1.el9 epel 1.3 M 2023-08-24T10:02:06.500 INFO:teuthology.orchestra.run.smithi060.stdout: jq x86_64 1.6-15.el9 CentOS-AppStream 186 k 2023-08-24T10:02:06.500 INFO:teuthology.orchestra.run.smithi060.stdout: libasan x86_64 11.4.1-2.1.el9 CentOS-AppStream 420 k 2023-08-24T10:02:06.500 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 161 k 2023-08-24T10:02:06.500 INFO:teuthology.orchestra.run.smithi060.stdout: libubsan x86_64 11.4.1-2.1.el9 CentOS-AppStream 189 k 2023-08-24T10:02:06.501 INFO:teuthology.orchestra.run.smithi060.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-08-24T10:02:06.501 INFO:teuthology.orchestra.run.smithi060.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-08-24T10:02:06.501 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-08-24T10:02:06.501 INFO:teuthology.orchestra.run.smithi060.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2023-08-24T10:02:06.501 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:02:06.501 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-08-24T10:02:06.501 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:02:06.502 INFO:teuthology.orchestra.run.smithi060.stdout:Install 10 Packages 2023-08-24T10:02:06.502 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:02:06.502 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 119 M 2023-08-24T10:02:06.503 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 1.1 G 2023-08-24T10:02:06.503 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-08-24T10:02:06.711 INFO:teuthology.orchestra.run.smithi060.stdout:(1/10): jq-1.6-15.el9.x86_64.rpm 1.2 MB/s | 186 kB 00:00 2023-08-24T10:02:06.761 INFO:teuthology.orchestra.run.smithi060.stdout:(2/10): libasan-11.4.1-2.1.el9.x86_64.rpm 8.2 MB/s | 420 kB 00:00 2023-08-24T10:02:06.795 INFO:teuthology.orchestra.run.smithi060.stdout:(3/10): libubsan-11.4.1-2.1.el9.x86_64.rpm 5.6 MB/s | 189 kB 00:00 2023-08-24T10:02:06.837 INFO:teuthology.orchestra.run.smithi060.stdout:(4/10): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 217 kB 00:00 2023-08-24T10:02:06.895 INFO:teuthology.orchestra.run.smithi060.stdout:(5/10): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2023-08-24T10:02:06.930 INFO:teuthology.orchestra.run.smithi060.stdout:(6/10): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2023-08-24T10:02:07.146 INFO:teuthology.orchestra.run.smithi060.stdout:(7/10): libcephsqlite-18.0.0-5670.g74f8c0ae.el9 275 kB/s | 161 kB 00:00 2023-08-24T10:02:07.455 INFO:teuthology.orchestra.run.smithi060.stdout:(8/10): cryptopp-8.6.0-1.el9.x86_64.rpm 2.4 MB/s | 1.3 MB 00:00 2023-08-24T10:02:07.480 INFO:teuthology.orchestra.run.smithi060.stdout:(9/10): yaml-cpp-0.6.3-5.el9.x86_64.rpm 365 kB/s | 122 kB 00:00 2023-08-24T10:02:07.851 INFO:teuthology.orchestra.run.smithi135.stdout:(10/10): ceph-test-18.0.0-5670.g74f8c0ae.el9.x8 22 MB/s | 116 MB 00:05 2023-08-24T10:02:07.855 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:02:07.856 INFO:teuthology.orchestra.run.smithi135.stdout:Total 23 MB/s | 119 MB 00:05 2023-08-24T10:02:07.856 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-08-24T10:02:07.878 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-08-24T10:02:07.878 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-08-24T10:02:08.692 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-08-24T10:02:08.693 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-08-24T10:02:09.817 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-08-24T10:02:09.915 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/10 2023-08-24T10:02:09.967 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cryptopp-8.6.0-1.el9.x86_64 2/10 2023-08-24T10:02:10.027 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/10 2023-08-24T10:02:10.077 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/10 2023-08-24T10:02:10.131 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 5/10 2023-08-24T10:02:10.166 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : jq-1.6-15.el9.x86_64 6/10 2023-08-24T10:02:10.203 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libubsan-11.4.1-2.1.el9.x86_64 7/10 2023-08-24T10:02:10.261 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libasan-11.4.1-2.1.el9.x86_64 8/10 2023-08-24T10:02:10.314 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephsqlite-2:18.0.0-5670.g74f8c0ae.el9.x86_64 9/10 2023-08-24T10:02:12.097 INFO:teuthology.orchestra.run.smithi060.stdout:(10/10): ceph-test-18.0.0-5670.g74f8c0ae.el9.x8 21 MB/s | 116 MB 00:05 2023-08-24T10:02:12.102 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:02:12.102 INFO:teuthology.orchestra.run.smithi060.stdout:Total 21 MB/s | 119 MB 00:05 2023-08-24T10:02:12.102 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-08-24T10:02:12.127 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-08-24T10:02:12.128 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-08-24T10:02:12.981 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-08-24T10:02:12.981 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-08-24T10:02:14.135 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-08-24T10:02:14.217 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/10 2023-08-24T10:02:14.246 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cryptopp-8.6.0-1.el9.x86_64 2/10 2023-08-24T10:02:14.296 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/10 2023-08-24T10:02:14.332 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/10 2023-08-24T10:02:14.366 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 5/10 2023-08-24T10:02:14.404 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : jq-1.6-15.el9.x86_64 6/10 2023-08-24T10:02:14.433 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libubsan-11.4.1-2.1.el9.x86_64 7/10 2023-08-24T10:02:14.475 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libasan-11.4.1-2.1.el9.x86_64 8/10 2023-08-24T10:02:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephsqlite-2:18.0.0-5670.g74f8c0ae.el9.x86_64 9/10 2023-08-24T10:02:26.383 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5670.g74f8c0ae.el9.x86_64 9/10 2023-08-24T10:02:26.469 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-test-2:18.0.0-5670.g74f8c0ae.el9.x86_64 10/10 2023-08-24T10:02:31.627 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5670.g74f8c0ae.el9.x86_64 9/10 2023-08-24T10:02:31.700 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-test-2:18.0.0-5670.g74f8c0ae.el9.x86_64 10/10 2023-08-24T10:02:36.444 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-test-2:18.0.0-5670.g74f8c0ae.el9.x86_64 10/10 2023-08-24T10:02:36.444 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-test-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/10 2023-08-24T10:02:36.444 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephsqlite-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2/10 2023-08-24T10:02:36.444 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : jq-1.6-15.el9.x86_64 3/10 2023-08-24T10:02:36.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libasan-11.4.1-2.1.el9.x86_64 4/10 2023-08-24T10:02:36.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libubsan-11.4.1-2.1.el9.x86_64 5/10 2023-08-24T10:02:36.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/10 2023-08-24T10:02:36.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 7/10 2023-08-24T10:02:36.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 8/10 2023-08-24T10:02:36.445 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cryptopp-8.6.0-1.el9.x86_64 9/10 2023-08-24T10:02:36.854 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 10/10 2023-08-24T10:02:36.854 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:02:36.855 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-08-24T10:02:36.855 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:36.855 INFO:teuthology.orchestra.run.smithi135.stdout: cryptopp-8.6.0-1.el9.x86_64 2023-08-24T10:02:36.855 INFO:teuthology.orchestra.run.smithi135.stdout: jq-1.6-15.el9.x86_64 2023-08-24T10:02:36.855 INFO:teuthology.orchestra.run.smithi135.stdout: libasan-11.4.1-2.1.el9.x86_64 2023-08-24T10:02:36.855 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:36.855 INFO:teuthology.orchestra.run.smithi135.stdout: libubsan-11.4.1-2.1.el9.x86_64 2023-08-24T10:02:36.856 INFO:teuthology.orchestra.run.smithi135.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-08-24T10:02:36.856 INFO:teuthology.orchestra.run.smithi135.stdout: socat-1.7.4.1-5.el9.x86_64 2023-08-24T10:02:36.856 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-08-24T10:02:36.856 INFO:teuthology.orchestra.run.smithi135.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2023-08-24T10:02:36.856 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:02:36.857 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:02:37.257 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph 2023-08-24T10:02:37.768 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:08 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:02:37.901 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:02:37.904 INFO:teuthology.orchestra.run.smithi135.stdout:========================================================================================= 2023-08-24T10:02:37.904 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-08-24T10:02:37.904 INFO:teuthology.orchestra.run.smithi135.stdout:========================================================================================= 2023-08-24T10:02:37.904 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-08-24T10:02:37.904 INFO:teuthology.orchestra.run.smithi135.stdout: ceph x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 6.5 k 2023-08-24T10:02:37.904 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-08-24T10:02:37.905 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 2.0 M 2023-08-24T10:02:37.905 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 1.4 M 2023-08-24T10:02:37.905 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5670.g74f8c0ae.el9 ceph-noarch 242 k 2023-08-24T10:02:37.905 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 8.8 M 2023-08-24T10:02:37.905 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 116 M 2023-08-24T10:02:37.905 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-08-24T10:02:37.905 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-08-24T10:02:37.906 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-08-24T10:02:37.906 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-08-24T10:02:37.906 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-08-24T10:02:37.906 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-08-24T10:02:37.906 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-08-24T10:02:37.906 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-08-24T10:02:37.906 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-08-24T10:02:37.906 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-08-24T10:02:37.907 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-08-24T10:02:37.907 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-08-24T10:02:37.907 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-08-24T10:02:37.907 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-08-24T10:02:37.907 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-08-24T10:02:37.907 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-08-24T10:02:37.908 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-08-24T10:02:37.908 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-08-24T10:02:37.908 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-08-24T10:02:37.908 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-08-24T10:02:37.909 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-08-24T10:02:37.909 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-08-24T10:02:37.909 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-08-24T10:02:37.909 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-08-24T10:02:37.909 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-08-24T10:02:37.909 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:02:37.909 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-08-24T10:02:37.909 INFO:teuthology.orchestra.run.smithi135.stdout:========================================================================================= 2023-08-24T10:02:37.910 INFO:teuthology.orchestra.run.smithi135.stdout:Install 31 Packages 2023-08-24T10:02:37.910 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:02:37.910 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 132 M 2023-08-24T10:02:37.911 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 1.4 G 2023-08-24T10:02:37.911 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-08-24T10:02:38.487 INFO:teuthology.orchestra.run.smithi135.stdout:(1/31): ceph-18.0.0-5670.g74f8c0ae.el9.x86_64.r 36 kB/s | 6.5 kB 00:00 2023-08-24T10:02:38.697 INFO:teuthology.orchestra.run.smithi135.stdout:(2/31): ceph-mds-18.0.0-5670.g74f8c0ae.el9.x86_ 5.2 MB/s | 2.0 MB 00:00 2023-08-24T10:02:38.745 INFO:teuthology.orchestra.run.smithi135.stdout:(3/31): ceph-mgr-18.0.0-5670.g74f8c0ae.el9.x86_ 3.3 MB/s | 1.4 MB 00:00 2023-08-24T10:02:38.838 INFO:teuthology.orchestra.run.smithi135.stdout:(4/31): ceph-mgr-modules-core-18.0.0-5670.g74f8 2.6 MB/s | 242 kB 00:00 2023-08-24T10:02:38.963 INFO:teuthology.orchestra.run.smithi135.stdout:(5/31): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 144 kB 00:00 2023-08-24T10:02:38.996 INFO:teuthology.orchestra.run.smithi135.stdout:(6/31): python3-markupsafe-1.1.1-12.el9.x86_64. 954 kB/s | 32 kB 00:00 2023-08-24T10:02:39.030 INFO:teuthology.orchestra.run.smithi135.stdout:(7/31): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2023-08-24T10:02:39.063 INFO:teuthology.orchestra.run.smithi135.stdout:(8/31): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2023-08-24T10:02:39.131 INFO:teuthology.orchestra.run.smithi135.stdout:(9/31): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2023-08-24T10:02:39.222 INFO:teuthology.orchestra.run.smithi135.stdout:(10/31): python3-cryptography-36.0.1-4.el9.x86_ 12 MB/s | 1.1 MB 00:00 2023-08-24T10:02:39.265 INFO:teuthology.orchestra.run.smithi135.stdout:(11/31): python3-ply-3.11-14.el9.noarch.rpm 2.4 MB/s | 103 kB 00:00 2023-08-24T10:02:39.315 INFO:teuthology.orchestra.run.smithi135.stdout:(12/31): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2023-08-24T10:02:39.723 INFO:teuthology.orchestra.run.smithi135.stdout:(13/31): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 106 kB/s | 43 kB 00:00 2023-08-24T10:02:40.015 INFO:teuthology.orchestra.run.smithi135.stdout:(14/31): python3-cheroot-8.6.0-4.el9.noarch.rpm 589 kB/s | 172 kB 00:00 2023-08-24T10:02:40.207 INFO:teuthology.orchestra.run.smithi135.stdout:(15/31): ceph-mon-18.0.0-5670.g74f8c0ae.el9.x86 5.1 MB/s | 8.8 MB 00:01 2023-08-24T10:02:40.266 INFO:teuthology.orchestra.run.smithi135.stdout:(16/31): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2023-08-24T10:02:40.300 INFO:teuthology.orchestra.run.smithi135.stdout:(17/31): python3-jaraco-8.2.1-3.el9.noarch.rpm 117 kB/s | 11 kB 00:00 2023-08-24T10:02:40.358 INFO:teuthology.orchestra.run.smithi135.stdout:(18/31): python3-jaraco-classes-3.2.1-5.el9.noa 192 kB/s | 18 kB 00:00 2023-08-24T10:02:40.400 INFO:teuthology.orchestra.run.smithi135.stdout:(19/31): python3-jaraco-collections-3.0.0-8.el9 231 kB/s | 23 kB 00:00 2023-08-24T10:02:40.459 INFO:teuthology.orchestra.run.smithi135.stdout:(20/31): python3-jaraco-functools-3.5.0-2.el9.n 194 kB/s | 19 kB 00:00 2023-08-24T10:02:40.501 INFO:teuthology.orchestra.run.smithi135.stdout:(21/31): python3-jaraco-text-3.2.0-6.el9.noarch 196 kB/s | 20 kB 00:00 2023-08-24T10:02:40.568 INFO:teuthology.orchestra.run.smithi135.stdout:(22/31): python3-logutils-0.3.5-21.el9.noarch.r 426 kB/s | 46 kB 00:00 2023-08-24T10:02:40.618 INFO:teuthology.orchestra.run.smithi135.stdout:(23/31): python3-more-itertools-8.12.0-2.el9.no 674 kB/s | 79 kB 00:00 2023-08-24T10:02:40.727 INFO:teuthology.orchestra.run.smithi135.stdout:(24/31): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2023-08-24T10:02:40.752 INFO:teuthology.orchestra.run.smithi135.stdout:(25/31): python3-portend-3.1.0-2.el9.noarch.rpm 123 kB/s | 16 kB 00:00 2023-08-24T10:02:40.894 INFO:teuthology.orchestra.run.smithi135.stdout:(26/31): python3-pyOpenSSL-21.0.0-1.el9.noarch. 539 kB/s | 90 kB 00:00 2023-08-24T10:02:40.919 INFO:teuthology.orchestra.run.smithi135.stdout:(27/31): python3-tempora-5.0.0-2.el9.noarch.rpm 214 kB/s | 36 kB 00:00 2023-08-24T10:02:41.028 INFO:teuthology.orchestra.run.smithi135.stdout:(28/31): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2023-08-24T10:02:41.087 INFO:teuthology.orchestra.run.smithi135.stdout:(29/31): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.5 MB/s | 427 kB 00:00 2023-08-24T10:02:41.128 INFO:teuthology.orchestra.run.smithi135.stdout:(30/31): python3-zc-lockfile-2.0-10.el9.noarch. 200 kB/s | 20 kB 00:00 2023-08-24T10:02:41.522 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-test-2:18.0.0-5670.g74f8c0ae.el9.x86_64 10/10 2023-08-24T10:02:41.522 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-test-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/10 2023-08-24T10:02:41.523 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephsqlite-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2/10 2023-08-24T10:02:41.523 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : jq-1.6-15.el9.x86_64 3/10 2023-08-24T10:02:41.523 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libasan-11.4.1-2.1.el9.x86_64 4/10 2023-08-24T10:02:41.523 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libubsan-11.4.1-2.1.el9.x86_64 5/10 2023-08-24T10:02:41.524 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/10 2023-08-24T10:02:41.524 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 7/10 2023-08-24T10:02:41.524 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 8/10 2023-08-24T10:02:41.525 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cryptopp-8.6.0-1.el9.x86_64 9/10 2023-08-24T10:02:41.947 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 10/10 2023-08-24T10:02:41.947 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:02:41.948 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-08-24T10:02:41.948 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:41.948 INFO:teuthology.orchestra.run.smithi060.stdout: cryptopp-8.6.0-1.el9.x86_64 2023-08-24T10:02:41.948 INFO:teuthology.orchestra.run.smithi060.stdout: jq-1.6-15.el9.x86_64 2023-08-24T10:02:41.948 INFO:teuthology.orchestra.run.smithi060.stdout: libasan-11.4.1-2.1.el9.x86_64 2023-08-24T10:02:41.948 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:02:41.948 INFO:teuthology.orchestra.run.smithi060.stdout: libubsan-11.4.1-2.1.el9.x86_64 2023-08-24T10:02:41.949 INFO:teuthology.orchestra.run.smithi060.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-08-24T10:02:41.949 INFO:teuthology.orchestra.run.smithi060.stdout: socat-1.7.4.1-5.el9.x86_64 2023-08-24T10:02:41.949 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-08-24T10:02:41.949 INFO:teuthology.orchestra.run.smithi060.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2023-08-24T10:02:41.949 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:02:41.949 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:02:42.308 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph 2023-08-24T10:02:42.842 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:08 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:02:42.980 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:02:42.983 INFO:teuthology.orchestra.run.smithi060.stdout:========================================================================================= 2023-08-24T10:02:42.983 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2023-08-24T10:02:42.983 INFO:teuthology.orchestra.run.smithi060.stdout:========================================================================================= 2023-08-24T10:02:42.984 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-08-24T10:02:42.984 INFO:teuthology.orchestra.run.smithi060.stdout: ceph x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 6.5 k 2023-08-24T10:02:42.984 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2023-08-24T10:02:42.984 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 2.0 M 2023-08-24T10:02:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 1.4 M 2023-08-24T10:02:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5670.g74f8c0ae.el9 ceph-noarch 242 k 2023-08-24T10:02:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 8.8 M 2023-08-24T10:02:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 116 M 2023-08-24T10:02:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-08-24T10:02:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-08-24T10:02:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-08-24T10:02:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-08-24T10:02:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-08-24T10:02:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-08-24T10:02:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-08-24T10:02:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-08-24T10:02:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-08-24T10:02:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-08-24T10:02:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-08-24T10:02:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-08-24T10:02:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-08-24T10:02:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-08-24T10:02:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-08-24T10:02:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-08-24T10:02:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-08-24T10:02:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-08-24T10:02:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-08-24T10:02:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-08-24T10:02:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-08-24T10:02:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-08-24T10:02:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-08-24T10:02:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-08-24T10:02:42.989 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-08-24T10:02:42.989 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:02:42.989 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-08-24T10:02:42.989 INFO:teuthology.orchestra.run.smithi060.stdout:========================================================================================= 2023-08-24T10:02:42.989 INFO:teuthology.orchestra.run.smithi060.stdout:Install 31 Packages 2023-08-24T10:02:42.989 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:02:42.990 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 132 M 2023-08-24T10:02:42.990 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 1.4 G 2023-08-24T10:02:42.990 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-08-24T10:02:43.526 INFO:teuthology.orchestra.run.smithi060.stdout:(1/31): ceph-18.0.0-5670.g74f8c0ae.el9.x86_64.r 31 kB/s | 6.5 kB 00:00 2023-08-24T10:02:43.659 INFO:teuthology.orchestra.run.smithi060.stdout:(2/31): ceph-mgr-18.0.0-5670.g74f8c0ae.el9.x86_ 4.2 MB/s | 1.4 MB 00:00 2023-08-24T10:02:44.009 INFO:teuthology.orchestra.run.smithi060.stdout:(3/31): ceph-mds-18.0.0-5670.g74f8c0ae.el9.x86_ 2.9 MB/s | 2.0 MB 00:00 2023-08-24T10:02:44.103 INFO:teuthology.orchestra.run.smithi135.stdout:(31/31): ceph-osd-18.0.0-5670.g74f8c0ae.el9.x86 21 MB/s | 116 MB 00:05 2023-08-24T10:02:44.108 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:02:44.108 INFO:teuthology.orchestra.run.smithi135.stdout:Total 21 MB/s | 132 MB 00:06 2023-08-24T10:02:44.109 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-08-24T10:02:44.143 INFO:teuthology.orchestra.run.smithi060.stdout:(4/31): ceph-mgr-modules-core-18.0.0-5670.g74f8 1.8 MB/s | 242 kB 00:00 2023-08-24T10:02:44.146 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-08-24T10:02:44.146 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-08-24T10:02:44.268 INFO:teuthology.orchestra.run.smithi060.stdout:(5/31): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 144 kB 00:00 2023-08-24T10:02:44.294 INFO:teuthology.orchestra.run.smithi060.stdout:(6/31): python3-markupsafe-1.1.1-12.el9.x86_64. 1.2 MB/s | 32 kB 00:00 2023-08-24T10:02:44.319 INFO:teuthology.orchestra.run.smithi060.stdout:(7/31): python3-pytz-2021.1-5.el9.noarch.rpm 1.9 MB/s | 47 kB 00:00 2023-08-24T10:02:44.344 INFO:teuthology.orchestra.run.smithi060.stdout:(8/31): python3-toml-0.10.2-6.el9.noarch.rpm 1.5 MB/s | 39 kB 00:00 2023-08-24T10:02:44.395 INFO:teuthology.orchestra.run.smithi060.stdout:(9/31): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2023-08-24T10:02:44.494 INFO:teuthology.orchestra.run.smithi060.stdout:(10/31): python3-cryptography-36.0.1-4.el9.x86_ 11 MB/s | 1.1 MB 00:00 2023-08-24T10:02:44.545 INFO:teuthology.orchestra.run.smithi060.stdout:(11/31): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2023-08-24T10:02:44.595 INFO:teuthology.orchestra.run.smithi060.stdout:(12/31): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2023-08-24T10:02:45.079 INFO:teuthology.orchestra.run.smithi060.stdout:(13/31): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 90 kB/s | 43 kB 00:00 2023-08-24T10:02:45.086 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-08-24T10:02:45.087 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-08-24T10:02:45.287 INFO:teuthology.orchestra.run.smithi060.stdout:(14/31): ceph-mon-18.0.0-5670.g74f8c0ae.el9.x86 5.0 MB/s | 8.8 MB 00:01 2023-08-24T10:02:45.438 INFO:teuthology.orchestra.run.smithi060.stdout:(15/31): python3-cheroot-8.6.0-4.el9.noarch.rpm 479 kB/s | 172 kB 00:00 2023-08-24T10:02:45.563 INFO:teuthology.orchestra.run.smithi060.stdout:(16/31): python3-jaraco-8.2.1-3.el9.noarch.rpm 85 kB/s | 11 kB 00:00 2023-08-24T10:02:45.688 INFO:teuthology.orchestra.run.smithi060.stdout:(17/31): python3-jaraco-classes-3.2.1-5.el9.noa 141 kB/s | 18 kB 00:00 2023-08-24T10:02:45.814 INFO:teuthology.orchestra.run.smithi060.stdout:(18/31): python3-jaraco-collections-3.0.0-8.el9 185 kB/s | 23 kB 00:00 2023-08-24T10:02:45.939 INFO:teuthology.orchestra.run.smithi060.stdout:(19/31): python3-jaraco-functools-3.5.0-2.el9.n 156 kB/s | 19 kB 00:00 2023-08-24T10:02:46.056 INFO:teuthology.orchestra.run.smithi060.stdout:(20/31): python3-cherrypy-18.6.1-2.el9.noarch.r 467 kB/s | 358 kB 00:00 2023-08-24T10:02:46.081 INFO:teuthology.orchestra.run.smithi060.stdout:(21/31): python3-jaraco-text-3.2.0-6.el9.noarch 139 kB/s | 20 kB 00:00 2023-08-24T10:02:46.181 INFO:teuthology.orchestra.run.smithi060.stdout:(22/31): python3-logutils-0.3.5-21.el9.noarch.r 370 kB/s | 46 kB 00:00 2023-08-24T10:02:46.240 INFO:teuthology.orchestra.run.smithi060.stdout:(23/31): python3-more-itertools-8.12.0-2.el9.no 497 kB/s | 79 kB 00:00 2023-08-24T10:02:46.366 INFO:teuthology.orchestra.run.smithi060.stdout:(24/31): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2023-08-24T10:02:46.391 INFO:teuthology.orchestra.run.smithi060.stdout:(25/31): python3-portend-3.1.0-2.el9.noarch.rpm 109 kB/s | 16 kB 00:00 2023-08-24T10:02:46.508 INFO:teuthology.orchestra.run.smithi060.stdout:(26/31): python3-pyOpenSSL-21.0.0-1.el9.noarch. 635 kB/s | 90 kB 00:00 2023-08-24T10:02:46.533 INFO:teuthology.orchestra.run.smithi060.stdout:(27/31): python3-tempora-5.0.0-2.el9.noarch.rpm 252 kB/s | 36 kB 00:00 2023-08-24T10:02:46.675 INFO:teuthology.orchestra.run.smithi060.stdout:(28/31): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2023-08-24T10:02:46.800 INFO:teuthology.orchestra.run.smithi060.stdout:(29/31): python3-zc-lockfile-2.0-10.el9.noarch. 160 kB/s | 20 kB 00:00 2023-08-24T10:02:46.810 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-08-24T10:02:46.842 INFO:teuthology.orchestra.run.smithi060.stdout:(30/31): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.4 MB/s | 427 kB 00:00 2023-08-24T10:02:46.897 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/31 2023-08-24T10:02:46.946 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/31 2023-08-24T10:02:46.987 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/31 2023-08-24T10:02:47.086 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/31 2023-08-24T10:02:47.174 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/31 2023-08-24T10:02:47.232 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/31 2023-08-24T10:02:47.286 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/31 2023-08-24T10:02:47.342 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ply-3.11-14.el9.noarch 8/31 2023-08-24T10:02:47.409 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 9/31 2023-08-24T10:02:47.612 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 10/31 2023-08-24T10:02:47.673 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 11/31 2023-08-24T10:02:47.717 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 12/31 2023-08-24T10:02:47.763 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 13/31 2023-08-24T10:02:47.801 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 14/31 2023-08-24T10:02:47.886 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 15/31 2023-08-24T10:02:47.936 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 16/31 2023-08-24T10:02:47.986 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 17/31 2023-08-24T10:02:48.044 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 18/31 2023-08-24T10:02:48.087 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/31 2023-08-24T10:02:48.122 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 20/31 2023-08-24T10:02:48.218 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 21/31 2023-08-24T10:02:48.269 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 22/31 2023-08-24T10:02:48.357 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/31 2023-08-24T10:02:48.510 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 24/31 2023-08-24T10:02:48.642 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 25/31 2023-08-24T10:02:48.812 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5670.g74f8c0ae.el9. 26/31 2023-08-24T10:02:48.849 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-2:18.0.0-5670.g74f8c0ae.el9.x86_64 27/31 2023-08-24T10:02:48.874 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5670.g74f8c0ae.el9.x86_64 27/31 2023-08-24T10:02:48.874 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-24T10:02:48.874 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-08-24T10:02:48.875 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-08-24T10:02:48.875 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-08-24T10:02:48.875 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:02:49.192 INFO:teuthology.orchestra.run.smithi060.stdout:(31/31): ceph-osd-18.0.0-5670.g74f8c0ae.el9.x86 21 MB/s | 116 MB 00:05 2023-08-24T10:02:49.199 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:02:49.199 INFO:teuthology.orchestra.run.smithi060.stdout:Total 21 MB/s | 132 MB 00:06 2023-08-24T10:02:49.200 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-08-24T10:02:49.241 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-08-24T10:02:49.242 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-08-24T10:02:50.231 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-08-24T10:02:50.231 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-08-24T10:02:52.017 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-08-24T10:02:52.095 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/31 2023-08-24T10:02:52.136 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/31 2023-08-24T10:02:52.177 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/31 2023-08-24T10:02:52.281 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/31 2023-08-24T10:02:52.374 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/31 2023-08-24T10:02:52.439 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/31 2023-08-24T10:02:52.485 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/31 2023-08-24T10:02:52.551 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ply-3.11-14.el9.noarch 8/31 2023-08-24T10:02:52.629 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 9/31 2023-08-24T10:02:52.849 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 10/31 2023-08-24T10:02:52.914 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 11/31 2023-08-24T10:02:52.961 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 12/31 2023-08-24T10:02:53.004 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 13/31 2023-08-24T10:02:53.050 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 14/31 2023-08-24T10:02:53.172 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 15/31 2023-08-24T10:02:53.219 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 16/31 2023-08-24T10:02:53.269 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 17/31 2023-08-24T10:02:53.318 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 18/31 2023-08-24T10:02:53.369 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/31 2023-08-24T10:02:53.412 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 20/31 2023-08-24T10:02:53.513 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 21/31 2023-08-24T10:02:53.576 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 22/31 2023-08-24T10:02:53.667 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/31 2023-08-24T10:02:53.824 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 24/31 2023-08-24T10:02:53.965 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 25/31 2023-08-24T10:02:54.144 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5670.g74f8c0ae.el9. 26/31 2023-08-24T10:02:54.189 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-2:18.0.0-5670.g74f8c0ae.el9.x86_64 27/31 2023-08-24T10:02:54.215 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5670.g74f8c0ae.el9.x86_64 27/31 2023-08-24T10:02:54.216 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-24T10:02:54.216 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-08-24T10:02:54.216 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-08-24T10:02:54.216 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-08-24T10:02:54.216 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:06.849 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-osd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 28/31 2023-08-24T10:03:06.874 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-osd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 28/31 2023-08-24T10:03:06.874 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-24T10:03:06.874 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-08-24T10:03:06.874 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-08-24T10:03:06.875 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-08-24T10:03:06.875 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:07.773 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mon-2:18.0.0-5670.g74f8c0ae.el9.x86_64 29/31 2023-08-24T10:03:07.796 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mon-2:18.0.0-5670.g74f8c0ae.el9.x86_64 29/31 2023-08-24T10:03:07.796 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-24T10:03:07.797 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-08-24T10:03:07.797 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-08-24T10:03:07.797 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-08-24T10:03:07.797 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:08.007 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mds-2:18.0.0-5670.g74f8c0ae.el9.x86_64 30/31 2023-08-24T10:03:08.030 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mds-2:18.0.0-5670.g74f8c0ae.el9.x86_64 30/31 2023-08-24T10:03:08.030 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-24T10:03:08.030 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-08-24T10:03:08.031 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-08-24T10:03:08.031 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-08-24T10:03:08.031 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:08.103 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-2:18.0.0-5670.g74f8c0ae.el9.x86_64 31/31 2023-08-24T10:03:13.422 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-osd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 28/31 2023-08-24T10:03:13.450 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-osd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 28/31 2023-08-24T10:03:13.450 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-24T10:03:13.450 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-08-24T10:03:13.450 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-08-24T10:03:13.450 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-08-24T10:03:13.451 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:14.389 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mon-2:18.0.0-5670.g74f8c0ae.el9.x86_64 29/31 2023-08-24T10:03:14.413 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mon-2:18.0.0-5670.g74f8c0ae.el9.x86_64 29/31 2023-08-24T10:03:14.413 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-24T10:03:14.413 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-08-24T10:03:14.414 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-08-24T10:03:14.414 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-08-24T10:03:14.414 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:14.648 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mds-2:18.0.0-5670.g74f8c0ae.el9.x86_64 30/31 2023-08-24T10:03:14.672 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mds-2:18.0.0-5670.g74f8c0ae.el9.x86_64 30/31 2023-08-24T10:03:14.673 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-24T10:03:14.673 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-08-24T10:03:14.673 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-08-24T10:03:14.673 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-08-24T10:03:14.673 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:14.756 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-2:18.0.0-5670.g74f8c0ae.el9.x86_64 31/31 2023-08-24T10:03:19.458 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-2:18.0.0-5670.g74f8c0ae.el9.x86_64 31/31 2023-08-24T10:03:19.458 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/31 2023-08-24T10:03:19.459 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mds-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2/31 2023-08-24T10:03:19.459 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-2:18.0.0-5670.g74f8c0ae.el9.x86_64 3/31 2023-08-24T10:03:19.459 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mon-2:18.0.0-5670.g74f8c0ae.el9.x86_64 4/31 2023-08-24T10:03:19.459 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-osd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 5/31 2023-08-24T10:03:19.459 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5670.g74f8c0ae.el9. 6/31 2023-08-24T10:03:19.459 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/31 2023-08-24T10:03:19.459 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/31 2023-08-24T10:03:19.460 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/31 2023-08-24T10:03:19.460 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/31 2023-08-24T10:03:19.460 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/31 2023-08-24T10:03:19.460 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/31 2023-08-24T10:03:19.460 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/31 2023-08-24T10:03:19.460 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/31 2023-08-24T10:03:19.460 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 15/31 2023-08-24T10:03:19.461 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 16/31 2023-08-24T10:03:19.461 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 17/31 2023-08-24T10:03:19.461 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 18/31 2023-08-24T10:03:19.461 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 19/31 2023-08-24T10:03:19.461 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 20/31 2023-08-24T10:03:19.461 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 21/31 2023-08-24T10:03:19.461 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 22/31 2023-08-24T10:03:19.461 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 23/31 2023-08-24T10:03:19.462 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 24/31 2023-08-24T10:03:19.462 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 25/31 2023-08-24T10:03:19.462 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 26/31 2023-08-24T10:03:19.462 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/31 2023-08-24T10:03:19.462 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 28/31 2023-08-24T10:03:19.462 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 29/31 2023-08-24T10:03:19.462 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 30/31 2023-08-24T10:03:20.502 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 31/31 2023-08-24T10:03:20.502 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:20.502 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-08-24T10:03:20.503 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:03:20.503 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:03:20.503 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:03:20.503 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core-2:18.0.0-5670.g74f8c0ae.el9.noarch 2023-08-24T10:03:20.503 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:03:20.503 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:03:20.503 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-08-24T10:03:20.503 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-08-24T10:03:20.504 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-08-24T10:03:20.504 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-08-24T10:03:20.504 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-08-24T10:03:20.504 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-08-24T10:03:20.504 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-08-24T10:03:20.504 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-08-24T10:03:20.504 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-08-24T10:03:20.505 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-08-24T10:03:20.505 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-08-24T10:03:20.505 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako-1.1.4-6.el9.noarch 2023-08-24T10:03:20.505 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-08-24T10:03:20.505 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-08-24T10:03:20.505 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-08-24T10:03:20.505 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply-3.11-14.el9.noarch 2023-08-24T10:03:20.505 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend-3.1.0-2.el9.noarch 2023-08-24T10:03:20.506 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-08-24T10:03:20.506 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser-2.20-6.el9.noarch 2023-08-24T10:03:20.506 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz-2021.1-5.el9.noarch 2023-08-24T10:03:20.506 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-08-24T10:03:20.506 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml-0.10.2-6.el9.noarch 2023-08-24T10:03:20.506 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob-1.8.7-6.el9.noarch 2023-08-24T10:03:20.506 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-08-24T10:03:20.506 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-08-24T10:03:20.507 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:20.507 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:03:20.756 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-base 2023-08-24T10:03:21.258 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:52 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:03:21.341 INFO:teuthology.orchestra.run.smithi135.stdout:Package ceph-base-2:18.0.0-5670.g74f8c0ae.el9.x86_64 is already installed. 2023-08-24T10:03:21.389 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:03:21.390 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-08-24T10:03:21.390 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:03:21.431 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install cephadm 2023-08-24T10:03:21.938 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:52 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:03:22.068 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:03:22.069 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:03:22.070 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-08-24T10:03:22.070 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:03:22.070 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-08-24T10:03:22.070 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm noarch 2:18.0.0-5670.g74f8c0ae.el9 ceph-noarch 225 k 2023-08-24T10:03:22.070 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:22.071 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-08-24T10:03:22.071 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:03:22.071 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2023-08-24T10:03:22.071 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:22.071 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 225 k 2023-08-24T10:03:22.071 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 217 k 2023-08-24T10:03:22.071 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-08-24T10:03:22.729 INFO:teuthology.orchestra.run.smithi135.stdout:cephadm-18.0.0-5670.g74f8c0ae.el9.noarch.rpm 342 kB/s | 225 kB 00:00 2023-08-24T10:03:22.730 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:03:22.730 INFO:teuthology.orchestra.run.smithi135.stdout:Total 341 kB/s | 225 kB 00:00 2023-08-24T10:03:22.730 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-08-24T10:03:22.735 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-08-24T10:03:22.735 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-08-24T10:03:22.738 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-08-24T10:03:22.738 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-08-24T10:03:22.807 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-08-24T10:03:23.256 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephadm-2:18.0.0-5670.g74f8c0ae.el9.noarch 1/1 2023-08-24T10:03:23.357 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephadm-2:18.0.0-5670.g74f8c0ae.el9.noarch 1/1 2023-08-24T10:03:23.714 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephadm-2:18.0.0-5670.g74f8c0ae.el9.noarch 1/1 2023-08-24T10:03:24.062 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephadm-2:18.0.0-5670.g74f8c0ae.el9.noarch 1/1 2023-08-24T10:03:24.062 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:24.062 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-08-24T10:03:24.063 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm-2:18.0.0-5670.g74f8c0ae.el9.noarch 2023-08-24T10:03:24.063 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:24.063 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:03:24.340 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-immutable-object-cache 2023-08-24T10:03:24.853 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:55 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:03:24.984 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:03:24.985 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:03:24.986 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repo Size 2023-08-24T10:03:24.986 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:03:24.986 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-08-24T10:03:24.986 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 144 k 2023-08-24T10:03:24.986 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:24.986 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-08-24T10:03:24.987 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:03:24.987 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2023-08-24T10:03:24.987 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:24.987 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 144 k 2023-08-24T10:03:24.987 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 438 k 2023-08-24T10:03:24.987 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-08-24T10:03:25.014 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-2:18.0.0-5670.g74f8c0ae.el9.x86_64 31/31 2023-08-24T10:03:25.015 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/31 2023-08-24T10:03:25.015 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mds-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2/31 2023-08-24T10:03:25.015 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-2:18.0.0-5670.g74f8c0ae.el9.x86_64 3/31 2023-08-24T10:03:25.015 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mon-2:18.0.0-5670.g74f8c0ae.el9.x86_64 4/31 2023-08-24T10:03:25.015 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-osd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 5/31 2023-08-24T10:03:25.015 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5670.g74f8c0ae.el9. 6/31 2023-08-24T10:03:25.016 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/31 2023-08-24T10:03:25.016 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/31 2023-08-24T10:03:25.016 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/31 2023-08-24T10:03:25.016 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/31 2023-08-24T10:03:25.016 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/31 2023-08-24T10:03:25.016 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/31 2023-08-24T10:03:25.016 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/31 2023-08-24T10:03:25.017 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/31 2023-08-24T10:03:25.017 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 15/31 2023-08-24T10:03:25.017 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 16/31 2023-08-24T10:03:25.017 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 17/31 2023-08-24T10:03:25.017 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 18/31 2023-08-24T10:03:25.017 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 19/31 2023-08-24T10:03:25.017 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 20/31 2023-08-24T10:03:25.017 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 21/31 2023-08-24T10:03:25.018 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 22/31 2023-08-24T10:03:25.018 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 23/31 2023-08-24T10:03:25.018 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 24/31 2023-08-24T10:03:25.018 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 25/31 2023-08-24T10:03:25.018 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 26/31 2023-08-24T10:03:25.019 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/31 2023-08-24T10:03:25.019 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 28/31 2023-08-24T10:03:25.019 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 29/31 2023-08-24T10:03:25.019 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 30/31 2023-08-24T10:03:25.205 INFO:teuthology.orchestra.run.smithi135.stdout:ceph-immutable-object-cache-18.0.0-5670.g74f8c0 671 kB/s | 144 kB 00:00 2023-08-24T10:03:25.206 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:03:25.206 INFO:teuthology.orchestra.run.smithi135.stdout:Total 666 kB/s | 144 kB 00:00 2023-08-24T10:03:25.206 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-08-24T10:03:25.211 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-08-24T10:03:25.211 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-08-24T10:03:25.265 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-08-24T10:03:25.266 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-08-24T10:03:25.424 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-08-24T10:03:25.536 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5670.g74f8c0ae. 1/1 2023-08-24T10:03:25.561 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5670.g74f8c0ae. 1/1 2023-08-24T10:03:25.561 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-24T10:03:25.561 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-08-24T10:03:25.561 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:26.311 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 31/31 2023-08-24T10:03:26.312 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:26.312 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-08-24T10:03:26.313 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:03:26.313 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:03:26.313 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:03:26.313 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core-2:18.0.0-5670.g74f8c0ae.el9.noarch 2023-08-24T10:03:26.314 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:03:26.314 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:03:26.314 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-08-24T10:03:26.314 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-08-24T10:03:26.315 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-08-24T10:03:26.316 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-08-24T10:03:26.316 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-08-24T10:03:26.317 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-08-24T10:03:26.317 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-08-24T10:03:26.317 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-08-24T10:03:26.317 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-08-24T10:03:26.318 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-08-24T10:03:26.318 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-08-24T10:03:26.318 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako-1.1.4-6.el9.noarch 2023-08-24T10:03:26.318 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-08-24T10:03:26.318 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-08-24T10:03:26.319 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-08-24T10:03:26.319 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply-3.11-14.el9.noarch 2023-08-24T10:03:26.319 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend-3.1.0-2.el9.noarch 2023-08-24T10:03:26.319 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-08-24T10:03:26.320 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser-2.20-6.el9.noarch 2023-08-24T10:03:26.320 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz-2021.1-5.el9.noarch 2023-08-24T10:03:26.320 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-08-24T10:03:26.321 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml-0.10.2-6.el9.noarch 2023-08-24T10:03:26.321 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob-1.8.7-6.el9.noarch 2023-08-24T10:03:26.321 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-08-24T10:03:26.321 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-08-24T10:03:26.321 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:26.321 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:03:26.580 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5670.g74f8c0ae. 1/1 2023-08-24T10:03:26.580 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:26.580 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-08-24T10:03:26.581 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:03:26.581 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:26.581 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:03:26.584 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-base 2023-08-24T10:03:26.845 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr 2023-08-24T10:03:27.116 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:53 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:03:27.207 INFO:teuthology.orchestra.run.smithi060.stdout:Package ceph-base-2:18.0.0-5670.g74f8c0ae.el9.x86_64 is already installed. 2023-08-24T10:03:27.253 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:03:27.254 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2023-08-24T10:03:27.254 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:03:27.296 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install cephadm 2023-08-24T10:03:27.368 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:58 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:03:27.457 INFO:teuthology.orchestra.run.smithi135.stdout:Package ceph-mgr-2:18.0.0-5670.g74f8c0ae.el9.x86_64 is already installed. 2023-08-24T10:03:27.506 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:03:27.507 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-08-24T10:03:27.507 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:03:27.563 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr-dashboard 2023-08-24T10:03:27.826 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:53 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:03:27.965 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:03:27.965 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:03:27.966 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2023-08-24T10:03:27.966 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:03:27.966 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-08-24T10:03:27.966 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm noarch 2:18.0.0-5670.g74f8c0ae.el9 ceph-noarch 225 k 2023-08-24T10:03:27.966 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:27.966 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-08-24T10:03:27.967 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:03:27.967 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2023-08-24T10:03:27.967 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:27.967 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 225 k 2023-08-24T10:03:27.967 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 217 k 2023-08-24T10:03:27.967 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-08-24T10:03:28.068 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:01:59 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:03:28.199 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:03:28.200 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:03:28.200 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-08-24T10:03:28.200 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:03:28.200 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-08-24T10:03:28.200 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5670.g74f8c0ae.el9 ceph-noarch 1.7 M 2023-08-24T10:03:28.200 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-08-24T10:03:28.201 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5670.g74f8c0ae.el9 ceph-noarch 23 k 2023-08-24T10:03:28.201 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5670.g74f8c0ae.el9 ceph-noarch 15 k 2023-08-24T10:03:28.201 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-08-24T10:03:28.201 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-08-24T10:03:28.201 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-08-24T10:03:28.201 INFO:teuthology.orchestra.run.smithi135.stdout:Installing weak dependencies: 2023-08-24T10:03:28.201 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-08-24T10:03:28.202 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:28.202 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-08-24T10:03:28.202 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:03:28.202 INFO:teuthology.orchestra.run.smithi135.stdout:Install 7 Packages 2023-08-24T10:03:28.202 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:28.203 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 2.0 M 2023-08-24T10:03:28.203 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 8.5 M 2023-08-24T10:03:28.203 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-08-24T10:03:28.330 INFO:teuthology.orchestra.run.smithi060.stdout:cephadm-18.0.0-5670.g74f8c0ae.el9.noarch.rpm 620 kB/s | 225 kB 00:00 2023-08-24T10:03:28.331 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:03:28.331 INFO:teuthology.orchestra.run.smithi060.stdout:Total 618 kB/s | 225 kB 00:00 2023-08-24T10:03:28.331 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-08-24T10:03:28.337 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-08-24T10:03:28.337 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-08-24T10:03:28.340 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-08-24T10:03:28.340 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-08-24T10:03:28.411 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-08-24T10:03:28.439 INFO:teuthology.orchestra.run.smithi135.stdout:(1/7): ceph-prometheus-alerts-18.0.0-5670.g74f8 76 kB/s | 15 kB 00:00 2023-08-24T10:03:28.539 INFO:teuthology.orchestra.run.smithi135.stdout:(2/7): ceph-grafana-dashboards-18.0.0-5670.g74f 79 kB/s | 23 kB 00:00 2023-08-24T10:03:28.597 INFO:teuthology.orchestra.run.smithi135.stdout:(3/7): ceph-mgr-dashboard-18.0.0-5670.g74f8c0ae 5.0 MB/s | 1.7 MB 00:00 2023-08-24T10:03:28.647 INFO:teuthology.orchestra.run.smithi135.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 43 kB/s | 9.0 kB 00:00 2023-08-24T10:03:28.747 INFO:teuthology.orchestra.run.smithi135.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 196 kB/s | 41 kB 00:00 2023-08-24T10:03:28.781 INFO:teuthology.orchestra.run.smithi135.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 191 kB/s | 31 kB 00:00 2023-08-24T10:03:28.869 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephadm-2:18.0.0-5670.g74f8c0ae.el9.noarch 1/1 2023-08-24T10:03:28.926 INFO:teuthology.orchestra.run.smithi135.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 749 kB/s | 188 kB 00:00 2023-08-24T10:03:28.926 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:03:28.926 INFO:teuthology.orchestra.run.smithi135.stdout:Total 2.8 MB/s | 2.0 MB 00:00 2023-08-24T10:03:28.926 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-08-24T10:03:28.934 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-08-24T10:03:28.934 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-08-24T10:03:28.953 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephadm-2:18.0.0-5670.g74f8c0ae.el9.noarch 1/1 2023-08-24T10:03:28.987 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-08-24T10:03:28.988 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-08-24T10:03:29.291 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-08-24T10:03:29.336 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephadm-2:18.0.0-5670.g74f8c0ae.el9.noarch 1/1 2023-08-24T10:03:29.374 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2023-08-24T10:03:29.416 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2023-08-24T10:03:29.479 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2023-08-24T10:03:29.528 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2023-08-24T10:03:29.565 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5670.g74f8c0ae.el9.n 5/7 2023-08-24T10:03:29.619 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephadm-2:18.0.0-5670.g74f8c0ae.el9.noarch 1/1 2023-08-24T10:03:29.620 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:29.620 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-08-24T10:03:29.620 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm-2:18.0.0-5670.g74f8c0ae.el9.noarch 2023-08-24T10:03:29.620 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:29.620 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:03:29.817 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5670.g74f8c0ae.el9. 6/7 2023-08-24T10:03:29.842 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-immutable-object-cache 2023-08-24T10:03:29.868 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5670.g74f8c0ae.el9.noarc 7/7 2023-08-24T10:03:30.379 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:56 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:03:30.484 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5670.g74f8c0ae.el9.noarc 7/7 2023-08-24T10:03:30.484 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5670.g74f8c0ae.el9. 1/7 2023-08-24T10:03:30.484 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5670.g74f8c0ae.el9.noarc 2/7 2023-08-24T10:03:30.484 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5670.g74f8c0ae.el9.n 3/7 2023-08-24T10:03:30.484 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2023-08-24T10:03:30.484 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2023-08-24T10:03:30.485 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2023-08-24T10:03:30.516 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:03:30.517 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:03:30.518 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repo Size 2023-08-24T10:03:30.518 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:03:30.518 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-08-24T10:03:30.519 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 144 k 2023-08-24T10:03:30.519 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:30.519 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-08-24T10:03:30.519 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:03:30.519 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2023-08-24T10:03:30.520 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:30.520 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 144 k 2023-08-24T10:03:30.520 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 438 k 2023-08-24T10:03:30.521 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-08-24T10:03:30.730 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-immutable-object-cache-18.0.0-5670.g74f8c0 684 kB/s | 144 kB 00:00 2023-08-24T10:03:30.730 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:03:30.731 INFO:teuthology.orchestra.run.smithi060.stdout:Total 679 kB/s | 144 kB 00:00 2023-08-24T10:03:30.731 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-08-24T10:03:30.739 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-08-24T10:03:30.739 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-08-24T10:03:30.796 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2023-08-24T10:03:30.797 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:30.797 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-08-24T10:03:30.797 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards-2:18.0.0-5670.g74f8c0ae.el9.noarch 2023-08-24T10:03:30.797 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard-2:18.0.0-5670.g74f8c0ae.el9.noarch 2023-08-24T10:03:30.797 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts-2:18.0.0-5670.g74f8c0ae.el9.noarch 2023-08-24T10:03:30.797 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-08-24T10:03:30.797 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-08-24T10:03:30.798 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-08-24T10:03:30.798 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes-2.5.1-5.el9.noarch 2023-08-24T10:03:30.798 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:30.798 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:03:30.799 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-08-24T10:03:30.799 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-08-24T10:03:30.952 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-08-24T10:03:31.023 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr-diskprediction-local 2023-08-24T10:03:31.039 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5670.g74f8c0ae. 1/1 2023-08-24T10:03:31.066 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5670.g74f8c0ae. 1/1 2023-08-24T10:03:31.066 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-24T10:03:31.066 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-08-24T10:03:31.066 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:31.531 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:02 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:03:31.662 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:03:31.663 INFO:teuthology.orchestra.run.smithi135.stdout:============================================================================================ 2023-08-24T10:03:31.664 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-08-24T10:03:31.664 INFO:teuthology.orchestra.run.smithi135.stdout:============================================================================================ 2023-08-24T10:03:31.664 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-08-24T10:03:31.664 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5670.g74f8c0ae.el9 ceph-noarch 7.4 M 2023-08-24T10:03:31.664 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-08-24T10:03:31.664 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-08-24T10:03:31.665 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-08-24T10:03:31.665 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-08-24T10:03:31.665 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran x86_64 11.4.1-2.1.el9 CentOS-BaseOS 807 k 2023-08-24T10:03:31.665 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath x86_64 11.4.1-2.1.el9 CentOS-BaseOS 197 k 2023-08-24T10:03:31.665 INFO:teuthology.orchestra.run.smithi135.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-08-24T10:03:31.665 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-08-24T10:03:31.665 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel x86_64 3.9.16-1.el9 appstream 247 k 2023-08-24T10:03:31.666 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-08-24T10:03:31.666 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-08-24T10:03:31.666 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-08-24T10:03:31.666 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:31.666 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-08-24T10:03:31.666 INFO:teuthology.orchestra.run.smithi135.stdout:============================================================================================ 2023-08-24T10:03:31.666 INFO:teuthology.orchestra.run.smithi135.stdout:Install 12 Packages 2023-08-24T10:03:31.666 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:31.667 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 37 M 2023-08-24T10:03:31.667 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 209 M 2023-08-24T10:03:31.667 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-08-24T10:03:32.072 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5670.g74f8c0ae. 1/1 2023-08-24T10:03:32.072 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:32.073 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-08-24T10:03:32.073 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:03:32.073 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:32.073 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:03:32.121 INFO:teuthology.orchestra.run.smithi135.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 233 kB/s | 32 kB 00:00 2023-08-24T10:03:32.156 INFO:teuthology.orchestra.run.smithi135.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 535 kB/s | 17 kB 00:00 2023-08-24T10:03:32.188 INFO:teuthology.orchestra.run.smithi135.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2023-08-24T10:03:32.297 INFO:teuthology.orchestra.run.smithi135.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.6 MB/s | 3.0 MB 00:00 2023-08-24T10:03:32.329 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-mgr 2023-08-24T10:03:32.439 INFO:teuthology.orchestra.run.smithi135.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 19 MB/s | 4.7 MB 00:00 2023-08-24T10:03:32.548 INFO:teuthology.orchestra.run.smithi135.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 4.8 MB 00:00 2023-08-24T10:03:32.581 INFO:teuthology.orchestra.run.smithi135.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.5 MB/s | 217 kB 00:00 2023-08-24T10:03:32.657 INFO:teuthology.orchestra.run.smithi135.stdout:(8/12): libgfortran-11.4.1-2.1.el9.x86_64.rpm 11 MB/s | 807 kB 00:00 2023-08-24T10:03:32.791 INFO:teuthology.orchestra.run.smithi135.stdout:(9/12): libquadmath-11.4.1-2.1.el9.x86_64.rpm 1.5 MB/s | 197 kB 00:00 2023-08-24T10:03:32.866 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:58 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:03:32.952 INFO:teuthology.orchestra.run.smithi060.stdout:Package ceph-mgr-2:18.0.0-5670.g74f8c0ae.el9.x86_64 is already installed. 2023-08-24T10:03:32.966 INFO:teuthology.orchestra.run.smithi135.stdout:(10/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2023-08-24T10:03:33.003 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:03:33.004 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2023-08-24T10:03:33.004 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:03:33.074 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-mgr-dashboard 2023-08-24T10:03:33.166 INFO:teuthology.orchestra.run.smithi135.stdout:(11/12): python3-devel-3.9.16-1.el9.x86_64.rpm 661 kB/s | 247 kB 00:00 2023-08-24T10:03:33.609 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:59 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:03:33.691 INFO:teuthology.orchestra.run.smithi135.stdout:(12/12): ceph-mgr-diskprediction-local-18.0.0-5 4.3 MB/s | 7.4 MB 00:01 2023-08-24T10:03:33.694 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:03:33.694 INFO:teuthology.orchestra.run.smithi135.stdout:Total 18 MB/s | 37 MB 00:02 2023-08-24T10:03:33.716 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-08-24T10:03:33.740 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-08-24T10:03:33.741 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-08-24T10:03:33.746 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:03:33.747 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:03:33.747 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2023-08-24T10:03:33.747 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:03:33.748 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-08-24T10:03:33.748 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5670.g74f8c0ae.el9 ceph-noarch 1.7 M 2023-08-24T10:03:33.748 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2023-08-24T10:03:33.748 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5670.g74f8c0ae.el9 ceph-noarch 23 k 2023-08-24T10:03:33.748 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5670.g74f8c0ae.el9 ceph-noarch 15 k 2023-08-24T10:03:33.748 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-08-24T10:03:33.748 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-08-24T10:03:33.749 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-08-24T10:03:33.749 INFO:teuthology.orchestra.run.smithi060.stdout:Installing weak dependencies: 2023-08-24T10:03:33.749 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-08-24T10:03:33.749 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:33.749 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-08-24T10:03:33.749 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:03:33.749 INFO:teuthology.orchestra.run.smithi060.stdout:Install 7 Packages 2023-08-24T10:03:33.749 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:33.750 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 2.0 M 2023-08-24T10:03:33.750 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 8.5 M 2023-08-24T10:03:33.750 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-08-24T10:03:34.101 INFO:teuthology.orchestra.run.smithi060.stdout:(1/7): ceph-grafana-dashboards-18.0.0-5670.g74f 117 kB/s | 23 kB 00:00 2023-08-24T10:03:34.151 INFO:teuthology.orchestra.run.smithi060.stdout:(2/7): ceph-prometheus-alerts-18.0.0-5670.g74f8 59 kB/s | 15 kB 00:00 2023-08-24T10:03:34.157 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-08-24T10:03:34.157 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-08-24T10:03:34.360 INFO:teuthology.orchestra.run.smithi060.stdout:(3/7): ceph-mgr-dashboard-18.0.0-5670.g74f8c0ae 3.8 MB/s | 1.7 MB 00:00 2023-08-24T10:03:34.777 INFO:teuthology.orchestra.run.smithi060.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 13 kB/s | 9.0 kB 00:00 2023-08-24T10:03:34.848 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-08-24T10:03:34.922 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libquadmath-11.4.1-2.1.el9.x86_64 1/12 2023-08-24T10:03:34.962 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libgfortran-11.4.1-2.1.el9.x86_64 2/12 2023-08-24T10:03:35.053 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2023-08-24T10:03:35.099 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-devel-3.9.16-1.el9.x86_64 4/12 2023-08-24T10:03:35.169 INFO:teuthology.orchestra.run.smithi060.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 40 kB/s | 41 kB 00:01 2023-08-24T10:03:35.287 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2023-08-24T10:03:35.323 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-08-24T10:03:35.327 INFO:teuthology.orchestra.run.smithi060.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 33 kB/s | 31 kB 00:00 2023-08-24T10:03:35.399 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2023-08-24T10:03:35.627 INFO:teuthology.orchestra.run.smithi060.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 222 kB/s | 188 kB 00:00 2023-08-24T10:03:35.634 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:03:35.634 INFO:teuthology.orchestra.run.smithi060.stdout:Total 1.1 MB/s | 2.0 MB 00:01 2023-08-24T10:03:35.634 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-08-24T10:03:35.643 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-08-24T10:03:35.643 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-08-24T10:03:35.709 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-08-24T10:03:35.709 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-08-24T10:03:36.020 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-08-24T10:03:36.073 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2023-08-24T10:03:36.104 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2023-08-24T10:03:36.155 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2023-08-24T10:03:36.195 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2023-08-24T10:03:36.228 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2023-08-24T10:03:36.275 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2023-08-24T10:03:36.312 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5670.g74f8c0ae.el9.n 5/7 2023-08-24T10:03:36.570 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5670.g74f8c0ae.el9. 6/7 2023-08-24T10:03:36.606 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5670.g74f8c0ae.el9.noarc 7/7 2023-08-24T10:03:37.182 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5670.g74f8c0ae.el9.noarc 7/7 2023-08-24T10:03:37.183 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5670.g74f8c0ae.el9. 1/7 2023-08-24T10:03:37.183 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5670.g74f8c0ae.el9.noarc 2/7 2023-08-24T10:03:37.183 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5670.g74f8c0ae.el9.n 3/7 2023-08-24T10:03:37.183 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2023-08-24T10:03:37.183 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2023-08-24T10:03:37.183 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2023-08-24T10:03:37.449 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2023-08-24T10:03:37.505 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2023-08-24T10:03:37.505 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:37.505 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-08-24T10:03:37.505 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards-2:18.0.0-5670.g74f8c0ae.el9.noarch 2023-08-24T10:03:37.506 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard-2:18.0.0-5670.g74f8c0ae.el9.noarch 2023-08-24T10:03:37.506 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts-2:18.0.0-5670.g74f8c0ae.el9.noarch 2023-08-24T10:03:37.506 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-08-24T10:03:37.506 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-08-24T10:03:37.506 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-08-24T10:03:37.506 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes-2.5.1-5.el9.noarch 2023-08-24T10:03:37.506 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:37.507 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:03:37.764 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-mgr-diskprediction-local 2023-08-24T10:03:38.298 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:02:04 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:03:38.440 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:03:38.441 INFO:teuthology.orchestra.run.smithi060.stdout:============================================================================================ 2023-08-24T10:03:38.441 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2023-08-24T10:03:38.442 INFO:teuthology.orchestra.run.smithi060.stdout:============================================================================================ 2023-08-24T10:03:38.442 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-08-24T10:03:38.442 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5670.g74f8c0ae.el9 ceph-noarch 7.4 M 2023-08-24T10:03:38.443 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2023-08-24T10:03:38.443 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-08-24T10:03:38.443 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-08-24T10:03:38.443 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-08-24T10:03:38.444 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran x86_64 11.4.1-2.1.el9 CentOS-BaseOS 807 k 2023-08-24T10:03:38.444 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath x86_64 11.4.1-2.1.el9 CentOS-BaseOS 197 k 2023-08-24T10:03:38.444 INFO:teuthology.orchestra.run.smithi060.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-08-24T10:03:38.445 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-08-24T10:03:38.445 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel x86_64 3.9.16-1.el9 appstream 247 k 2023-08-24T10:03:38.445 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-08-24T10:03:38.445 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-08-24T10:03:38.446 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-08-24T10:03:38.446 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:38.446 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-08-24T10:03:38.446 INFO:teuthology.orchestra.run.smithi060.stdout:============================================================================================ 2023-08-24T10:03:38.447 INFO:teuthology.orchestra.run.smithi060.stdout:Install 12 Packages 2023-08-24T10:03:38.447 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:38.448 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 37 M 2023-08-24T10:03:38.448 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 209 M 2023-08-24T10:03:38.448 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-08-24T10:03:38.487 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2023-08-24T10:03:38.522 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5670.g74f8c 12/12 2023-08-24T10:03:38.618 INFO:teuthology.orchestra.run.smithi060.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 242 kB/s | 32 kB 00:00 2023-08-24T10:03:38.652 INFO:teuthology.orchestra.run.smithi060.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 518 kB/s | 17 kB 00:00 2023-08-24T10:03:38.694 INFO:teuthology.orchestra.run.smithi060.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 825 kB/s | 34 kB 00:00 2023-08-24T10:03:38.786 INFO:teuthology.orchestra.run.smithi060.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 10 MB/s | 3.0 MB 00:00 2023-08-24T10:03:38.961 INFO:teuthology.orchestra.run.smithi060.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-08-24T10:03:39.061 INFO:teuthology.orchestra.run.smithi060.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 4.8 MB 00:00 2023-08-24T10:03:39.095 INFO:teuthology.orchestra.run.smithi060.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.6 MB/s | 217 kB 00:00 2023-08-24T10:03:39.195 INFO:teuthology.orchestra.run.smithi060.stdout:(8/12): libgfortran-11.4.1-2.1.el9.x86_64.rpm 8.1 MB/s | 807 kB 00:00 2023-08-24T10:03:39.429 INFO:teuthology.orchestra.run.smithi060.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2023-08-24T10:03:39.454 INFO:teuthology.orchestra.run.smithi060.stdout:(10/12): libquadmath-11.4.1-2.1.el9.x86_64.rpm 768 kB/s | 197 kB 00:00 2023-08-24T10:03:40.046 INFO:teuthology.orchestra.run.smithi060.stdout:(11/12): python3-devel-3.9.16-1.el9.x86_64.rpm 401 kB/s | 247 kB 00:00 2023-08-24T10:03:40.838 INFO:teuthology.orchestra.run.smithi060.stdout:(12/12): ceph-mgr-diskprediction-local-18.0.0-5 3.2 MB/s | 7.4 MB 00:02 2023-08-24T10:03:40.842 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:03:40.843 INFO:teuthology.orchestra.run.smithi060.stdout:Total 15 MB/s | 37 MB 00:02 2023-08-24T10:03:40.860 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-08-24T10:03:40.889 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-08-24T10:03:40.889 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-08-24T10:03:41.046 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5670.g74f8c 12/12 2023-08-24T10:03:41.046 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5670.g74f8c 1/12 2023-08-24T10:03:41.046 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2023-08-24T10:03:41.047 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2023-08-24T10:03:41.047 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2023-08-24T10:03:41.047 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2023-08-24T10:03:41.047 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-08-24T10:03:41.047 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2023-08-24T10:03:41.047 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2023-08-24T10:03:41.047 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2023-08-24T10:03:41.048 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libgfortran-11.4.1-2.1.el9.x86_64 10/12 2023-08-24T10:03:41.048 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libquadmath-11.4.1-2.1.el9.x86_64 11/12 2023-08-24T10:03:41.301 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-08-24T10:03:41.304 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-08-24T10:03:41.482 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-devel-3.9.16-1.el9.x86_64 12/12 2023-08-24T10:03:41.482 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:41.482 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-08-24T10:03:41.482 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5670.g74f8c0ae.el9.noarch 2023-08-24T10:03:41.482 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-08-24T10:03:41.482 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-08-24T10:03:41.483 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-08-24T10:03:41.483 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran-11.4.1-2.1.el9.x86_64 2023-08-24T10:03:41.483 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath-11.4.1-2.1.el9.x86_64 2023-08-24T10:03:41.483 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-0.3.21-2.el9.x86_64 2023-08-24T10:03:41.483 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-08-24T10:03:41.483 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel-3.9.16-1.el9.x86_64 2023-08-24T10:03:41.483 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-08-24T10:03:41.484 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-08-24T10:03:41.484 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-08-24T10:03:41.484 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:41.484 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:03:41.905 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr-rook 2023-08-24T10:03:42.022 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-08-24T10:03:42.086 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libquadmath-11.4.1-2.1.el9.x86_64 1/12 2023-08-24T10:03:42.125 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libgfortran-11.4.1-2.1.el9.x86_64 2/12 2023-08-24T10:03:42.223 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2023-08-24T10:03:42.262 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-devel-3.9.16-1.el9.x86_64 4/12 2023-08-24T10:03:42.426 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:13 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:03:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2023-08-24T10:03:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-08-24T10:03:42.556 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2023-08-24T10:03:42.558 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:03:42.559 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================== 2023-08-24T10:03:42.559 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-08-24T10:03:42.559 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================== 2023-08-24T10:03:42.560 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-08-24T10:03:42.560 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook noarch 2:18.0.0-5670.g74f8c0ae.el9 ceph-noarch 47 k 2023-08-24T10:03:42.560 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-08-24T10:03:42.560 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-08-24T10:03:42.560 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-08-24T10:03:42.560 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth noarch 1:2.22.0-1.el9 epel 197 k 2023-08-24T10:03:42.560 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-08-24T10:03:42.561 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-08-24T10:03:42.561 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-08-24T10:03:42.561 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-08-24T10:03:42.562 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-08-24T10:03:42.562 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-08-24T10:03:42.562 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-08-24T10:03:42.562 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-08-24T10:03:42.563 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-08-24T10:03:42.563 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:42.563 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-08-24T10:03:42.564 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================== 2023-08-24T10:03:42.564 INFO:teuthology.orchestra.run.smithi135.stdout:Install 13 Packages 2023-08-24T10:03:42.564 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:42.566 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 2.0 M 2023-08-24T10:03:42.566 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 26 M 2023-08-24T10:03:42.566 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-08-24T10:03:42.872 INFO:teuthology.orchestra.run.smithi135.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 187 kB/s | 24 kB 00:00 2023-08-24T10:03:42.897 INFO:teuthology.orchestra.run.smithi135.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 116 kB/s | 18 kB 00:00 2023-08-24T10:03:42.955 INFO:teuthology.orchestra.run.smithi135.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.1 MB/s | 174 kB 00:00 2023-08-24T10:03:42.989 INFO:teuthology.orchestra.run.smithi135.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 132 kB 00:00 2023-08-24T10:03:43.023 INFO:teuthology.orchestra.run.smithi135.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.1 MB/s | 211 kB 00:00 2023-08-24T10:03:43.048 INFO:teuthology.orchestra.run.smithi135.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 732 kB/s | 43 kB 00:00 2023-08-24T10:03:43.081 INFO:teuthology.orchestra.run.smithi135.stdout:(7/13): ceph-mgr-rook-18.0.0-5670.g74f8c0ae.el9 140 kB/s | 47 kB 00:00 2023-08-24T10:03:43.263 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2023-08-24T10:03:43.357 INFO:teuthology.orchestra.run.smithi135.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 49 kB/s | 15 kB 00:00 2023-08-24T10:03:43.377 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2023-08-24T10:03:43.407 INFO:teuthology.orchestra.run.smithi135.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 84 kB/s | 32 kB 00:00 2023-08-24T10:03:43.582 INFO:teuthology.orchestra.run.smithi135.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 337 kB/s | 59 kB 00:00 2023-08-24T10:03:43.658 INFO:teuthology.orchestra.run.smithi135.stdout:(11/13): python3-google-auth-2.22.0-1.el9.noarc 341 kB/s | 197 kB 00:00 2023-08-24T10:03:43.725 INFO:teuthology.orchestra.run.smithi135.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 631 kB/s | 90 kB 00:00 2023-08-24T10:03:43.875 INFO:teuthology.orchestra.run.smithi135.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.0 MB/s | 1.0 MB 00:00 2023-08-24T10:03:43.879 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:03:43.880 INFO:teuthology.orchestra.run.smithi135.stdout:Total 1.5 MB/s | 2.0 MB 00:01 2023-08-24T10:03:43.881 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-08-24T10:03:43.893 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-08-24T10:03:43.894 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-08-24T10:03:43.980 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-08-24T10:03:43.984 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-08-24T10:03:44.489 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-08-24T10:03:44.679 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2023-08-24T10:03:44.695 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2023-08-24T10:03:44.759 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2023-08-24T10:03:44.832 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2023-08-24T10:03:44.885 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2023-08-24T10:03:44.935 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2023-08-24T10:03:45.036 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2023-08-24T10:03:45.145 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-google-auth-1:2.22.0-1.el9.noarch 7/13 2023-08-24T10:03:45.203 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2023-08-24T10:03:45.775 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2023-08-24T10:03:45.802 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5670.g74f8c 12/12 2023-08-24T10:03:45.940 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2023-08-24T10:03:46.015 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2023-08-24T10:03:46.066 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2023-08-24T10:03:46.121 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2023-08-24T10:03:46.153 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-rook-2:18.0.0-5670.g74f8c0ae.el9.noarch 13/13 2023-08-24T10:03:47.020 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5670.g74f8c0ae.el9.noarch 13/13 2023-08-24T10:03:47.020 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5670.g74f8c0ae.el9.noarch 1/13 2023-08-24T10:03:47.020 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2023-08-24T10:03:47.021 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2023-08-24T10:03:47.021 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2023-08-24T10:03:47.021 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2023-08-24T10:03:47.021 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2023-08-24T10:03:47.021 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2023-08-24T10:03:47.022 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2023-08-24T10:03:47.022 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2023-08-24T10:03:47.022 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-google-auth-1:2.22.0-1.el9.noarch 10/13 2023-08-24T10:03:47.022 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2023-08-24T10:03:47.023 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2023-08-24T10:03:47.418 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2023-08-24T10:03:47.419 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:47.419 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-08-24T10:03:47.419 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook-2:18.0.0-5670.g74f8c0ae.el9.noarch 2023-08-24T10:03:47.419 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-08-24T10:03:47.420 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-08-24T10:03:47.420 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth-1:2.22.0-1.el9.noarch 2023-08-24T10:03:47.421 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-08-24T10:03:47.421 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-08-24T10:03:47.421 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-08-24T10:03:47.422 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-08-24T10:03:47.422 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-08-24T10:03:47.423 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-08-24T10:03:47.423 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-08-24T10:03:47.423 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa-4.9-2.el9.noarch 2023-08-24T10:03:47.423 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-08-24T10:03:47.424 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:47.424 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:03:47.684 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-mgr-cephadm 2023-08-24T10:03:48.197 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:19 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:03:48.332 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:03:48.332 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================== 2023-08-24T10:03:48.333 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-08-24T10:03:48.333 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================== 2023-08-24T10:03:48.333 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-08-24T10:03:48.333 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5670.g74f8c0ae.el9 ceph-noarch 132 k 2023-08-24T10:03:48.333 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-08-24T10:03:48.333 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-08-24T10:03:48.333 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-08-24T10:03:48.334 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-08-24T10:03:48.334 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-08-24T10:03:48.334 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-08-24T10:03:48.334 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:48.334 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-08-24T10:03:48.334 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================== 2023-08-24T10:03:48.334 INFO:teuthology.orchestra.run.smithi135.stdout:Install 6 Packages 2023-08-24T10:03:48.335 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:48.335 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 6.8 M 2023-08-24T10:03:48.335 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 33 M 2023-08-24T10:03:48.335 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-08-24T10:03:48.392 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5670.g74f8c 12/12 2023-08-24T10:03:48.392 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5670.g74f8c 1/12 2023-08-24T10:03:48.392 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2023-08-24T10:03:48.393 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2023-08-24T10:03:48.393 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2023-08-24T10:03:48.393 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2023-08-24T10:03:48.393 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-08-24T10:03:48.393 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2023-08-24T10:03:48.393 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2023-08-24T10:03:48.393 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2023-08-24T10:03:48.394 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libgfortran-11.4.1-2.1.el9.x86_64 10/12 2023-08-24T10:03:48.394 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libquadmath-11.4.1-2.1.el9.x86_64 11/12 2023-08-24T10:03:48.681 INFO:teuthology.orchestra.run.smithi135.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 1.4 MB/s | 228 kB 00:00 2023-08-24T10:03:48.840 INFO:teuthology.orchestra.run.smithi135.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 5.8 MB 00:00 2023-08-24T10:03:48.868 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-devel-3.9.16-1.el9.x86_64 12/12 2023-08-24T10:03:48.868 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:48.869 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-08-24T10:03:48.869 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5670.g74f8c0ae.el9.noarch 2023-08-24T10:03:48.870 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-08-24T10:03:48.870 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-08-24T10:03:48.871 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-08-24T10:03:48.871 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran-11.4.1-2.1.el9.x86_64 2023-08-24T10:03:48.871 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath-11.4.1-2.1.el9.x86_64 2023-08-24T10:03:48.871 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-0.3.21-2.el9.x86_64 2023-08-24T10:03:48.872 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-08-24T10:03:48.872 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel-3.9.16-1.el9.x86_64 2023-08-24T10:03:48.872 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-08-24T10:03:48.872 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-08-24T10:03:48.873 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-08-24T10:03:48.874 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:48.874 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:03:49.198 INFO:teuthology.orchestra.run.smithi135.stdout:(3/6): ceph-mgr-cephadm-18.0.0-5670.g74f8c0ae.e 195 kB/s | 132 kB 00:00 2023-08-24T10:03:49.315 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-mgr-rook 2023-08-24T10:03:49.340 INFO:teuthology.orchestra.run.smithi135.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 121 kB/s | 58 kB 00:00 2023-08-24T10:03:49.549 INFO:teuthology.orchestra.run.smithi135.stdout:(5/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 632 kB/s | 548 kB 00:00 2023-08-24T10:03:49.641 INFO:teuthology.orchestra.run.smithi135.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 115 kB/s | 51 kB 00:00 2023-08-24T10:03:49.644 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:03:49.645 INFO:teuthology.orchestra.run.smithi135.stdout:Total 5.2 MB/s | 6.8 MB 00:01 2023-08-24T10:03:49.645 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-08-24T10:03:49.654 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-08-24T10:03:49.654 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-08-24T10:03:49.738 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-08-24T10:03:49.739 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-08-24T10:03:49.863 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:02:15 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:03:50.002 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:03:50.003 INFO:teuthology.orchestra.run.smithi060.stdout:======================================================================================== 2023-08-24T10:03:50.003 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2023-08-24T10:03:50.003 INFO:teuthology.orchestra.run.smithi060.stdout:======================================================================================== 2023-08-24T10:03:50.003 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-08-24T10:03:50.003 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook noarch 2:18.0.0-5670.g74f8c0ae.el9 ceph-noarch 47 k 2023-08-24T10:03:50.004 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2023-08-24T10:03:50.004 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-08-24T10:03:50.004 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-08-24T10:03:50.004 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth noarch 1:2.22.0-1.el9 epel 197 k 2023-08-24T10:03:50.004 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-08-24T10:03:50.004 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-08-24T10:03:50.005 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-08-24T10:03:50.005 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-08-24T10:03:50.005 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-08-24T10:03:50.005 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-08-24T10:03:50.005 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-08-24T10:03:50.005 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-08-24T10:03:50.005 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-08-24T10:03:50.005 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:50.006 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-08-24T10:03:50.006 INFO:teuthology.orchestra.run.smithi060.stdout:======================================================================================== 2023-08-24T10:03:50.006 INFO:teuthology.orchestra.run.smithi060.stdout:Install 13 Packages 2023-08-24T10:03:50.006 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:50.006 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 2.0 M 2023-08-24T10:03:50.007 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 26 M 2023-08-24T10:03:50.007 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-08-24T10:03:50.026 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-08-24T10:03:50.195 INFO:teuthology.orchestra.run.smithi060.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 180 kB/s | 24 kB 00:00 2023-08-24T10:03:50.203 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2023-08-24T10:03:50.212 INFO:teuthology.orchestra.run.smithi060.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 119 kB/s | 18 kB 00:00 2023-08-24T10:03:50.267 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2023-08-24T10:03:50.271 INFO:teuthology.orchestra.run.smithi060.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2023-08-24T10:03:50.297 INFO:teuthology.orchestra.run.smithi060.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2023-08-24T10:03:50.321 INFO:teuthology.orchestra.run.smithi060.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2023-08-24T10:03:50.338 INFO:teuthology.orchestra.run.smithi060.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 43 kB 00:00 2023-08-24T10:03:50.472 INFO:teuthology.orchestra.run.smithi060.stdout:(7/13): python3-certifi-2021.10.8-2.el9.noarch. 112 kB/s | 15 kB 00:00 2023-08-24T10:03:50.489 INFO:teuthology.orchestra.run.smithi060.stdout:(8/13): ceph-mgr-rook-18.0.0-5670.g74f8c0ae.el9 111 kB/s | 47 kB 00:00 2023-08-24T10:03:50.514 INFO:teuthology.orchestra.run.smithi060.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 167 kB/s | 32 kB 00:00 2023-08-24T10:03:50.608 INFO:teuthology.orchestra.run.smithi060.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 643 kB/s | 59 kB 00:00 2023-08-24T10:03:50.632 INFO:teuthology.orchestra.run.smithi060.stdout:(11/13): python3-google-auth-2.22.0-1.el9.noarc 1.2 MB/s | 197 kB 00:00 2023-08-24T10:03:50.707 INFO:teuthology.orchestra.run.smithi060.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 4.7 MB/s | 1.0 MB 00:00 2023-08-24T10:03:50.735 INFO:teuthology.orchestra.run.smithi060.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 715 kB/s | 90 kB 00:00 2023-08-24T10:03:50.737 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:03:50.737 INFO:teuthology.orchestra.run.smithi060.stdout:Total 2.8 MB/s | 2.0 MB 00:00 2023-08-24T10:03:50.737 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-08-24T10:03:50.750 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-08-24T10:03:50.751 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-08-24T10:03:50.792 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2023-08-24T10:03:50.846 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-08-24T10:03:50.846 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-08-24T10:03:50.893 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2023-08-24T10:03:50.971 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2023-08-24T10:03:51.005 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5670.g74f8c0ae.el9.noarch 6/6 2023-08-24T10:03:51.360 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-08-24T10:03:51.540 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2023-08-24T10:03:51.618 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2023-08-24T10:03:51.683 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2023-08-24T10:03:51.727 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2023-08-24T10:03:51.776 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2023-08-24T10:03:51.846 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5670.g74f8c0ae.el9.noarch 6/6 2023-08-24T10:03:51.846 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5670.g74f8c0ae.el9.noarch 1/6 2023-08-24T10:03:51.846 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2023-08-24T10:03:51.846 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2023-08-24T10:03:51.847 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2023-08-24T10:03:51.847 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2023-08-24T10:03:51.889 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2023-08-24T10:03:52.016 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-google-auth-1:2.22.0-1.el9.noarch 7/13 2023-08-24T10:03:52.087 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2023-08-24T10:03:52.166 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2023-08-24T10:03:52.167 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:52.167 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-08-24T10:03:52.167 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm-2:18.0.0-5670.g74f8c0ae.el9.noarch 2023-08-24T10:03:52.167 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-08-24T10:03:52.167 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel-2.9.1-2.el9.noarch 2023-08-24T10:03:52.168 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-08-24T10:03:52.168 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-08-24T10:03:52.168 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-08-24T10:03:52.168 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:52.168 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:03:52.456 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-fuse 2023-08-24T10:03:52.857 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2023-08-24T10:03:52.926 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2023-08-24T10:03:52.966 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:23 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:03:52.969 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2023-08-24T10:03:53.024 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2023-08-24T10:03:53.055 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-rook-2:18.0.0-5670.g74f8c0ae.el9.noarch 13/13 2023-08-24T10:03:53.098 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:03:53.098 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:03:53.098 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-08-24T10:03:53.099 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:03:53.099 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-08-24T10:03:53.099 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 824 k 2023-08-24T10:03:53.099 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2023-08-24T10:03:53.100 INFO:teuthology.orchestra.run.smithi135.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-08-24T10:03:53.100 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:53.100 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-08-24T10:03:53.100 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:03:53.100 INFO:teuthology.orchestra.run.smithi135.stdout:Install 2 Packages 2023-08-24T10:03:53.101 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:53.102 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 903 k 2023-08-24T10:03:53.102 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 2.9 M 2023-08-24T10:03:53.102 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-08-24T10:03:53.234 INFO:teuthology.orchestra.run.smithi135.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 585 kB/s | 79 kB 00:00 2023-08-24T10:03:53.576 INFO:teuthology.orchestra.run.smithi135.stdout:(2/2): ceph-fuse-18.0.0-5670.g74f8c0ae.el9.x86_ 1.7 MB/s | 824 kB 00:00 2023-08-24T10:03:53.577 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:03:53.577 INFO:teuthology.orchestra.run.smithi135.stdout:Total 1.8 MB/s | 903 kB 00:00 2023-08-24T10:03:53.578 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-08-24T10:03:53.599 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-08-24T10:03:53.600 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-08-24T10:03:53.674 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-08-24T10:03:53.675 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-08-24T10:03:53.854 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5670.g74f8c0ae.el9.noarch 13/13 2023-08-24T10:03:53.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5670.g74f8c0ae.el9.noarch 1/13 2023-08-24T10:03:53.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2023-08-24T10:03:53.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2023-08-24T10:03:53.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2023-08-24T10:03:53.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2023-08-24T10:03:53.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2023-08-24T10:03:53.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2023-08-24T10:03:53.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2023-08-24T10:03:53.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2023-08-24T10:03:53.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-google-auth-1:2.22.0-1.el9.noarch 10/13 2023-08-24T10:03:53.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2023-08-24T10:03:53.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2023-08-24T10:03:53.864 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-08-24T10:03:54.021 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2023-08-24T10:03:54.107 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-fuse-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2/2 2023-08-24T10:03:54.231 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2023-08-24T10:03:54.231 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:54.231 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-08-24T10:03:54.231 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook-2:18.0.0-5670.g74f8c0ae.el9.noarch 2023-08-24T10:03:54.232 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-08-24T10:03:54.232 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-08-24T10:03:54.232 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth-1:2.22.0-1.el9.noarch 2023-08-24T10:03:54.232 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-08-24T10:03:54.232 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-08-24T10:03:54.232 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-08-24T10:03:54.233 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-08-24T10:03:54.233 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-08-24T10:03:54.233 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-08-24T10:03:54.233 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-08-24T10:03:54.233 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa-4.9-2.el9.noarch 2023-08-24T10:03:54.233 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-08-24T10:03:54.233 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:54.233 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:03:54.505 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-mgr-cephadm 2023-08-24T10:03:54.845 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-fuse-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2/2 2023-08-24T10:03:54.845 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-fuse-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/2 2023-08-24T10:03:55.045 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:02:21 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:03:55.158 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2023-08-24T10:03:55.158 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:55.158 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-08-24T10:03:55.159 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse-2:18.0.0-5670.g74f8c0ae.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2023-08-24T10:03:55.159 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:55.159 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:03:55.186 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:03:55.186 INFO:teuthology.orchestra.run.smithi060.stdout:======================================================================================== 2023-08-24T10:03:55.186 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2023-08-24T10:03:55.187 INFO:teuthology.orchestra.run.smithi060.stdout:======================================================================================== 2023-08-24T10:03:55.187 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-08-24T10:03:55.187 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5670.g74f8c0ae.el9 ceph-noarch 132 k 2023-08-24T10:03:55.187 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2023-08-24T10:03:55.187 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-08-24T10:03:55.187 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-08-24T10:03:55.187 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-08-24T10:03:55.188 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-08-24T10:03:55.188 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-08-24T10:03:55.188 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:55.188 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-08-24T10:03:55.188 INFO:teuthology.orchestra.run.smithi060.stdout:======================================================================================== 2023-08-24T10:03:55.188 INFO:teuthology.orchestra.run.smithi060.stdout:Install 6 Packages 2023-08-24T10:03:55.188 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:55.189 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 6.8 M 2023-08-24T10:03:55.189 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 33 M 2023-08-24T10:03:55.189 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-08-24T10:03:55.412 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-volume 2023-08-24T10:03:55.516 INFO:teuthology.orchestra.run.smithi060.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 1.4 MB/s | 228 kB 00:00 2023-08-24T10:03:55.691 INFO:teuthology.orchestra.run.smithi060.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2023-08-24T10:03:55.750 INFO:teuthology.orchestra.run.smithi060.stdout:(3/6): ceph-mgr-cephadm-18.0.0-5670.g74f8c0ae.e 333 kB/s | 132 kB 00:00 2023-08-24T10:03:55.938 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:26 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:03:56.073 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:03:56.074 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:03:56.074 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2023-08-24T10:03:56.074 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:03:56.075 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-08-24T10:03:56.075 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume noarch 2:18.0.0-5670.g74f8c0ae.el9 ceph-noarch 261 k 2023-08-24T10:03:56.075 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:56.075 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-08-24T10:03:56.075 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:03:56.075 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2023-08-24T10:03:56.075 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:56.076 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 261 k 2023-08-24T10:03:56.076 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 1.2 M 2023-08-24T10:03:56.076 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-08-24T10:03:56.317 INFO:teuthology.orchestra.run.smithi060.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 96 kB/s | 58 kB 00:00 2023-08-24T10:03:56.342 INFO:teuthology.orchestra.run.smithi060.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 86 kB/s | 51 kB 00:00 2023-08-24T10:03:56.485 INFO:teuthology.orchestra.run.smithi135.stdout:ceph-volume-18.0.0-5670.g74f8c0ae.el9.noarch.rp 645 kB/s | 261 kB 00:00 2023-08-24T10:03:56.485 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:03:56.485 INFO:teuthology.orchestra.run.smithi135.stdout:Total 643 kB/s | 261 kB 00:00 2023-08-24T10:03:56.486 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-08-24T10:03:56.490 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-08-24T10:03:56.490 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-08-24T10:03:56.522 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-08-24T10:03:56.523 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-08-24T10:03:56.734 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-08-24T10:03:56.807 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-volume-2:18.0.0-5670.g74f8c0ae.el9.noarch 1/1 2023-08-24T10:03:56.823 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-volume-2:18.0.0-5670.g74f8c0ae.el9.noarch 1/1 2023-08-24T10:03:56.823 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-24T10:03:56.823 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-08-24T10:03:56.824 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:56.851 INFO:teuthology.orchestra.run.smithi060.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 411 kB/s | 548 kB 00:01 2023-08-24T10:03:56.855 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:03:56.855 INFO:teuthology.orchestra.run.smithi060.stdout:Total 4.1 MB/s | 6.8 MB 00:01 2023-08-24T10:03:56.860 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-08-24T10:03:56.866 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-08-24T10:03:56.866 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-08-24T10:03:56.960 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-08-24T10:03:56.961 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-08-24T10:03:57.256 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-08-24T10:03:57.448 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2023-08-24T10:03:57.520 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2023-08-24T10:03:57.821 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-volume-2:18.0.0-5670.g74f8c0ae.el9.noarch 1/1 2023-08-24T10:03:57.821 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:57.822 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-08-24T10:03:57.822 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume-2:18.0.0-5670.g74f8c0ae.el9.noarch 2023-08-24T10:03:57.822 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:57.822 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:03:58.094 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2023-08-24T10:03:58.150 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install librados-devel 2023-08-24T10:03:58.199 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2023-08-24T10:03:58.275 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2023-08-24T10:03:58.306 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5670.g74f8c0ae.el9.noarch 6/6 2023-08-24T10:03:58.668 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:29 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:03:58.803 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:03:58.804 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:03:58.804 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repo Size 2023-08-24T10:03:58.804 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:03:58.804 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-08-24T10:03:58.804 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 127 k 2023-08-24T10:03:58.805 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:58.805 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-08-24T10:03:58.805 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:03:58.805 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2023-08-24T10:03:58.805 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:03:58.805 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 127 k 2023-08-24T10:03:58.805 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 460 k 2023-08-24T10:03:58.805 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-08-24T10:03:59.074 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5670.g74f8c0ae.el9.noarch 6/6 2023-08-24T10:03:59.074 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5670.g74f8c0ae.el9.noarch 1/6 2023-08-24T10:03:59.074 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2023-08-24T10:03:59.075 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2023-08-24T10:03:59.075 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2023-08-24T10:03:59.076 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2023-08-24T10:03:59.344 INFO:teuthology.orchestra.run.smithi135.stdout:librados-devel-18.0.0-5670.g74f8c0ae.el9.x86_64 235 kB/s | 127 kB 00:00 2023-08-24T10:03:59.344 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:03:59.345 INFO:teuthology.orchestra.run.smithi135.stdout:Total 235 kB/s | 127 kB 00:00 2023-08-24T10:03:59.345 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-08-24T10:03:59.350 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-08-24T10:03:59.351 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-08-24T10:03:59.403 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2023-08-24T10:03:59.404 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:59.404 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-08-24T10:03:59.404 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm-2:18.0.0-5670.g74f8c0ae.el9.noarch 2023-08-24T10:03:59.404 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-08-24T10:03:59.404 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel-2.9.1-2.el9.noarch 2023-08-24T10:03:59.405 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-08-24T10:03:59.405 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-08-24T10:03:59.405 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-08-24T10:03:59.405 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:03:59.405 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:03:59.410 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-08-24T10:03:59.410 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-08-24T10:03:59.561 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-08-24T10:03:59.684 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librados-devel-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:03:59.711 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-fuse 2023-08-24T10:04:00.103 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados-devel-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:00.247 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:02:26 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:04:00.387 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:04:00.388 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:00.388 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2023-08-24T10:04:00.388 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:00.388 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-08-24T10:04:00.388 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 824 k 2023-08-24T10:04:00.389 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2023-08-24T10:04:00.389 INFO:teuthology.orchestra.run.smithi060.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-08-24T10:04:00.389 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:00.389 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-08-24T10:04:00.389 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:00.389 INFO:teuthology.orchestra.run.smithi060.stdout:Install 2 Packages 2023-08-24T10:04:00.389 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:00.390 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 903 k 2023-08-24T10:04:00.390 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 2.9 M 2023-08-24T10:04:00.390 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-08-24T10:04:00.432 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados-devel-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:00.432 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:00.432 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-08-24T10:04:00.433 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:04:00.433 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:00.433 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:04:00.495 INFO:teuthology.orchestra.run.smithi060.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 746 kB/s | 79 kB 00:00 2023-08-24T10:04:00.710 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install libcephfs2 2023-08-24T10:04:01.003 INFO:teuthology.orchestra.run.smithi060.stdout:(2/2): ceph-fuse-18.0.0-5670.g74f8c0ae.el9.x86_ 1.3 MB/s | 824 kB 00:00 2023-08-24T10:04:01.005 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:04:01.005 INFO:teuthology.orchestra.run.smithi060.stdout:Total 1.4 MB/s | 903 kB 00:00 2023-08-24T10:04:01.006 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-08-24T10:04:01.029 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-08-24T10:04:01.029 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-08-24T10:04:01.110 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-08-24T10:04:01.115 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-08-24T10:04:01.230 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:32 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:04:01.314 INFO:teuthology.orchestra.run.smithi135.stdout:Package libcephfs2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 is already installed. 2023-08-24T10:04:01.325 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-08-24T10:04:01.363 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:04:01.365 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-08-24T10:04:01.365 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:04:01.445 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install libcephfs-devel 2023-08-24T10:04:01.484 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2023-08-24T10:04:01.570 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-fuse-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2/2 2023-08-24T10:04:01.959 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:32 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:04:02.091 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:04:02.091 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:04:02.092 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repo Size 2023-08-24T10:04:02.092 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:04:02.092 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-08-24T10:04:02.092 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 31 k 2023-08-24T10:04:02.092 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:02.092 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-08-24T10:04:02.093 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:04:02.093 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2023-08-24T10:04:02.093 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:02.093 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 31 k 2023-08-24T10:04:02.093 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 140 k 2023-08-24T10:04:02.093 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-08-24T10:04:02.331 INFO:teuthology.orchestra.run.smithi135.stdout:libcephfs-devel-18.0.0-5670.g74f8c0ae.el9.x86_6 131 kB/s | 31 kB 00:00 2023-08-24T10:04:02.332 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:04:02.332 INFO:teuthology.orchestra.run.smithi135.stdout:Total 130 kB/s | 31 kB 00:00 2023-08-24T10:04:02.332 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-08-24T10:04:02.335 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-08-24T10:04:02.335 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-08-24T10:04:02.349 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-08-24T10:04:02.350 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-08-24T10:04:02.362 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-fuse-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2/2 2023-08-24T10:04:02.363 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-fuse-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/2 2023-08-24T10:04:02.431 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-08-24T10:04:02.561 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs-devel-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:02.687 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2023-08-24T10:04:02.687 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:02.687 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-08-24T10:04:02.687 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse-2:18.0.0-5670.g74f8c0ae.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2023-08-24T10:04:02.687 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:02.688 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:04:02.915 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:03.007 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-volume 2023-08-24T10:04:03.180 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs-devel-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:03.180 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:03.181 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-08-24T10:04:03.181 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:04:03.181 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:03.182 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:04:03.398 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install librados2 2023-08-24T10:04:03.551 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:02:29 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:04:03.689 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:04:03.691 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:03.691 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2023-08-24T10:04:03.691 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:03.692 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-08-24T10:04:03.692 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume noarch 2:18.0.0-5670.g74f8c0ae.el9 ceph-noarch 261 k 2023-08-24T10:04:03.692 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:03.693 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-08-24T10:04:03.693 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:03.693 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2023-08-24T10:04:03.694 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:03.694 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 261 k 2023-08-24T10:04:03.694 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 1.2 M 2023-08-24T10:04:03.694 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-08-24T10:04:03.910 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:34 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:04:03.992 INFO:teuthology.orchestra.run.smithi135.stdout:Package librados2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 is already installed. 2023-08-24T10:04:04.034 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-volume-18.0.0-5670.g74f8c0ae.el9.noarch.rp 762 kB/s | 261 kB 00:00 2023-08-24T10:04:04.035 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:04:04.035 INFO:teuthology.orchestra.run.smithi060.stdout:Total 759 kB/s | 261 kB 00:00 2023-08-24T10:04:04.035 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-08-24T10:04:04.041 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:04:04.042 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-08-24T10:04:04.042 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:04:04.044 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-08-24T10:04:04.044 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-08-24T10:04:04.078 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-08-24T10:04:04.078 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-08-24T10:04:04.104 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install librbd1 2023-08-24T10:04:04.313 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-08-24T10:04:04.411 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-volume-2:18.0.0-5670.g74f8c0ae.el9.noarch 1/1 2023-08-24T10:04:04.427 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-volume-2:18.0.0-5670.g74f8c0ae.el9.noarch 1/1 2023-08-24T10:04:04.428 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-24T10:04:04.428 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-08-24T10:04:04.428 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:04.617 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:35 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:04:04.700 INFO:teuthology.orchestra.run.smithi135.stdout:Package librbd1-2:18.0.0-5670.g74f8c0ae.el9.x86_64 is already installed. 2023-08-24T10:04:04.749 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:04:04.750 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-08-24T10:04:04.751 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:04:04.813 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install python3-rados 2023-08-24T10:04:05.325 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:36 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:04:05.408 INFO:teuthology.orchestra.run.smithi135.stdout:Package python3-rados-2:18.0.0-5670.g74f8c0ae.el9.x86_64 is already installed. 2023-08-24T10:04:05.456 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:04:05.457 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-08-24T10:04:05.458 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:04:05.492 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-volume-2:18.0.0-5670.g74f8c0ae.el9.noarch 1/1 2023-08-24T10:04:05.492 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:05.492 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-08-24T10:04:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume-2:18.0.0-5670.g74f8c0ae.el9.noarch 2023-08-24T10:04:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:05.493 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:04:05.527 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install python3-rgw 2023-08-24T10:04:05.864 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install librados-devel 2023-08-24T10:04:06.038 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:37 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:04:06.122 INFO:teuthology.orchestra.run.smithi135.stdout:Package python3-rgw-2:18.0.0-5670.g74f8c0ae.el9.x86_64 is already installed. 2023-08-24T10:04:06.171 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:04:06.172 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-08-24T10:04:06.172 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:04:06.236 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install python3-cephfs 2023-08-24T10:04:06.407 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:02:32 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:04:06.546 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:04:06.547 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:06.547 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repo Size 2023-08-24T10:04:06.547 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:06.547 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-08-24T10:04:06.548 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 127 k 2023-08-24T10:04:06.548 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:06.548 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-08-24T10:04:06.548 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:06.548 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2023-08-24T10:04:06.548 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:06.548 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 127 k 2023-08-24T10:04:06.549 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 460 k 2023-08-24T10:04:06.549 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-08-24T10:04:06.749 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:37 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:04:06.765 INFO:teuthology.orchestra.run.smithi060.stdout:librados-devel-18.0.0-5670.g74f8c0ae.el9.x86_64 585 kB/s | 127 kB 00:00 2023-08-24T10:04:06.766 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:04:06.766 INFO:teuthology.orchestra.run.smithi060.stdout:Total 580 kB/s | 127 kB 00:00 2023-08-24T10:04:06.767 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-08-24T10:04:06.772 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-08-24T10:04:06.773 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-08-24T10:04:06.832 INFO:teuthology.orchestra.run.smithi135.stdout:Package python3-cephfs-2:18.0.0-5670.g74f8c0ae.el9.x86_64 is already installed. 2023-08-24T10:04:06.834 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-08-24T10:04:06.835 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-08-24T10:04:06.881 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:04:06.882 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-08-24T10:04:06.882 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:04:06.947 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install python3-rbd 2023-08-24T10:04:06.988 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-08-24T10:04:07.115 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librados-devel-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:07.457 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:38 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:04:07.546 INFO:teuthology.orchestra.run.smithi135.stdout:Package python3-rbd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 is already installed. 2023-08-24T10:04:07.565 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados-devel-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:07.589 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:04:07.590 INFO:teuthology.orchestra.run.smithi135.stdout:Nothing to do. 2023-08-24T10:04:07.590 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:04:07.656 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install rbd-fuse 2023-08-24T10:04:07.912 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados-devel-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:07.913 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:07.913 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-08-24T10:04:07.913 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:04:07.913 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:07.913 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:04:08.163 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:39 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:04:08.175 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install libcephfs2 2023-08-24T10:04:08.295 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:04:08.297 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:04:08.297 INFO:teuthology.orchestra.run.smithi135.stdout: Package Architecture Version Repository Size 2023-08-24T10:04:08.297 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:04:08.297 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-08-24T10:04:08.297 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 87 k 2023-08-24T10:04:08.297 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:08.298 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-08-24T10:04:08.298 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:04:08.298 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2023-08-24T10:04:08.298 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:08.298 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 87 k 2023-08-24T10:04:08.298 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 252 k 2023-08-24T10:04:08.298 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-08-24T10:04:08.517 INFO:teuthology.orchestra.run.smithi135.stdout:rbd-fuse-18.0.0-5670.g74f8c0ae.el9.x86_64.rpm 394 kB/s | 87 kB 00:00 2023-08-24T10:04:08.519 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:04:08.519 INFO:teuthology.orchestra.run.smithi135.stdout:Total 392 kB/s | 87 kB 00:00 2023-08-24T10:04:08.519 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-08-24T10:04:08.525 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-08-24T10:04:08.526 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-08-24T10:04:08.587 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-08-24T10:04:08.588 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-08-24T10:04:08.718 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:02:34 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:04:08.735 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-08-24T10:04:08.806 INFO:teuthology.orchestra.run.smithi060.stdout:Package libcephfs2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 is already installed. 2023-08-24T10:04:08.831 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-fuse-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:08.859 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:04:08.860 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2023-08-24T10:04:08.860 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:04:08.939 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install libcephfs-devel 2023-08-24T10:04:09.224 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-fuse-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:09.479 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:02:35 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:04:09.559 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-fuse-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:09.559 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:09.559 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-08-24T10:04:09.560 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:04:09.560 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:09.560 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:04:09.620 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:04:09.622 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:09.622 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repo Size 2023-08-24T10:04:09.623 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:09.623 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-08-24T10:04:09.623 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 31 k 2023-08-24T10:04:09.623 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:09.623 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-08-24T10:04:09.623 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:09.623 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2023-08-24T10:04:09.624 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:09.624 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 31 k 2023-08-24T10:04:09.624 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 140 k 2023-08-24T10:04:09.624 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-08-24T10:04:09.821 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install rbd-mirror 2023-08-24T10:04:09.854 INFO:teuthology.orchestra.run.smithi060.stdout:libcephfs-devel-18.0.0-5670.g74f8c0ae.el9.x86_6 134 kB/s | 31 kB 00:00 2023-08-24T10:04:09.854 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:04:09.855 INFO:teuthology.orchestra.run.smithi060.stdout:Total 134 kB/s | 31 kB 00:00 2023-08-24T10:04:09.855 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-08-24T10:04:09.858 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-08-24T10:04:09.858 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-08-24T10:04:09.879 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-08-24T10:04:09.879 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-08-24T10:04:09.962 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-08-24T10:04:10.069 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs-devel-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:10.344 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:41 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:04:10.479 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:04:10.481 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:04:10.482 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repo Size 2023-08-24T10:04:10.482 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:04:10.482 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-08-24T10:04:10.482 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 3.0 M 2023-08-24T10:04:10.483 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:10.483 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-08-24T10:04:10.483 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:04:10.484 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2023-08-24T10:04:10.484 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:10.484 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 3.0 M 2023-08-24T10:04:10.484 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 13 M 2023-08-24T10:04:10.485 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-08-24T10:04:10.515 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:10.776 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs-devel-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:10.776 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:10.776 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-08-24T10:04:10.777 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:04:10.777 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:10.777 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:04:11.013 INFO:teuthology.orchestra.run.smithi135.stdout:rbd-mirror-18.0.0-5670.g74f8c0ae.el9.x86_64.rpm 5.6 MB/s | 3.0 MB 00:00 2023-08-24T10:04:11.013 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:04:11.014 INFO:teuthology.orchestra.run.smithi135.stdout:Total 5.6 MB/s | 3.0 MB 00:00 2023-08-24T10:04:11.014 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-08-24T10:04:11.021 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install librados2 2023-08-24T10:04:11.023 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-08-24T10:04:11.023 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-08-24T10:04:11.099 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-08-24T10:04:11.100 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-08-24T10:04:11.521 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-08-24T10:04:11.556 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:02:37 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:04:11.580 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-mirror-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:11.605 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:11.605 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-24T10:04:11.605 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-08-24T10:04:11.605 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-08-24T10:04:11.606 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-08-24T10:04:11.606 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:11.641 INFO:teuthology.orchestra.run.smithi060.stdout:Package librados2-2:18.0.0-5670.g74f8c0ae.el9.x86_64 is already installed. 2023-08-24T10:04:11.694 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:04:11.695 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2023-08-24T10:04:11.695 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:04:11.768 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install librbd1 2023-08-24T10:04:12.308 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:02:38 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:04:12.395 INFO:teuthology.orchestra.run.smithi060.stdout:Package librbd1-2:18.0.0-5670.g74f8c0ae.el9.x86_64 is already installed. 2023-08-24T10:04:12.449 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:04:12.450 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2023-08-24T10:04:12.450 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:04:12.516 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install python3-rados 2023-08-24T10:04:12.656 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-mirror-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:12.657 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:12.657 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-08-24T10:04:12.657 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:04:12.658 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:12.658 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:04:12.893 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install rbd-nbd 2023-08-24T10:04:13.057 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:02:39 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:04:13.144 INFO:teuthology.orchestra.run.smithi060.stdout:Package python3-rados-2:18.0.0-5670.g74f8c0ae.el9.x86_64 is already installed. 2023-08-24T10:04:13.197 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:04:13.198 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2023-08-24T10:04:13.198 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:04:13.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install python3-rgw 2023-08-24T10:04:13.412 INFO:teuthology.orchestra.run.smithi135.stdout:Last metadata expiration check: 0:02:44 ago on Thu 24 Aug 2023 10:01:29 AM UTC. 2023-08-24T10:04:13.542 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2023-08-24T10:04:13.543 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:04:13.543 INFO:teuthology.orchestra.run.smithi135.stdout: Package Architecture Version Repository Size 2023-08-24T10:04:13.543 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:04:13.543 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2023-08-24T10:04:13.543 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 168 k 2023-08-24T10:04:13.543 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:13.543 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2023-08-24T10:04:13.544 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2023-08-24T10:04:13.544 INFO:teuthology.orchestra.run.smithi135.stdout:Install 1 Package 2023-08-24T10:04:13.544 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:13.544 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 168 k 2023-08-24T10:04:13.544 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 533 k 2023-08-24T10:04:13.544 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2023-08-24T10:04:13.800 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:02:39 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:04:13.887 INFO:teuthology.orchestra.run.smithi060.stdout:Package python3-rgw-2:18.0.0-5670.g74f8c0ae.el9.x86_64 is already installed. 2023-08-24T10:04:13.939 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:04:13.940 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2023-08-24T10:04:13.940 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:04:14.006 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install python3-cephfs 2023-08-24T10:04:14.133 INFO:teuthology.orchestra.run.smithi135.stdout:rbd-nbd-18.0.0-5670.g74f8c0ae.el9.x86_64.rpm 286 kB/s | 168 kB 00:00 2023-08-24T10:04:14.134 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:04:14.134 INFO:teuthology.orchestra.run.smithi135.stdout:Total 285 kB/s | 168 kB 00:00 2023-08-24T10:04:14.135 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2023-08-24T10:04:14.143 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2023-08-24T10:04:14.143 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2023-08-24T10:04:14.201 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2023-08-24T10:04:14.201 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2023-08-24T10:04:14.347 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2023-08-24T10:04:14.479 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-nbd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:14.542 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:02:40 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:04:14.628 INFO:teuthology.orchestra.run.smithi060.stdout:Package python3-cephfs-2:18.0.0-5670.g74f8c0ae.el9.x86_64 is already installed. 2023-08-24T10:04:14.683 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:04:14.684 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2023-08-24T10:04:14.684 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:04:14.746 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install python3-rbd 2023-08-24T10:04:14.856 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-nbd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:15.170 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-nbd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:15.170 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:15.171 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2023-08-24T10:04:15.171 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:04:15.171 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:15.171 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2023-08-24T10:04:15.284 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:02:41 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:04:15.373 INFO:teuthology.orchestra.run.smithi060.stdout:Package python3-rbd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 is already installed. 2023-08-24T10:04:15.425 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:04:15.426 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2023-08-24T10:04:15.426 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:04:15.442 DEBUG:teuthology.parallel:result is None 2023-08-24T10:04:15.492 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install rbd-fuse 2023-08-24T10:04:16.032 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:02:42 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:04:16.172 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:04:16.173 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:16.173 INFO:teuthology.orchestra.run.smithi060.stdout: Package Architecture Version Repository Size 2023-08-24T10:04:16.174 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:16.174 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-08-24T10:04:16.174 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 87 k 2023-08-24T10:04:16.174 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:16.174 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-08-24T10:04:16.174 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:16.175 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2023-08-24T10:04:16.175 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:16.175 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 87 k 2023-08-24T10:04:16.175 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 252 k 2023-08-24T10:04:16.175 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-08-24T10:04:16.382 INFO:teuthology.orchestra.run.smithi060.stdout:rbd-fuse-18.0.0-5670.g74f8c0ae.el9.x86_64.rpm 418 kB/s | 87 kB 00:00 2023-08-24T10:04:16.382 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:04:16.383 INFO:teuthology.orchestra.run.smithi060.stdout:Total 415 kB/s | 87 kB 00:00 2023-08-24T10:04:16.383 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-08-24T10:04:16.390 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-08-24T10:04:16.391 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-08-24T10:04:16.456 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-08-24T10:04:16.457 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-08-24T10:04:16.611 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-08-24T10:04:16.757 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-fuse-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:17.180 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-fuse-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:17.494 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-fuse-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:17.495 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:17.495 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-08-24T10:04:17.495 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:04:17.495 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:17.495 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:04:17.760 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install rbd-mirror 2023-08-24T10:04:18.302 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:02:44 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:04:18.439 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:04:18.440 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:18.441 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repo Size 2023-08-24T10:04:18.441 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:18.441 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-08-24T10:04:18.441 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 3.0 M 2023-08-24T10:04:18.441 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:18.441 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-08-24T10:04:18.441 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:18.441 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2023-08-24T10:04:18.442 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:18.442 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 3.0 M 2023-08-24T10:04:18.442 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 13 M 2023-08-24T10:04:18.442 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-08-24T10:04:18.929 INFO:teuthology.orchestra.run.smithi060.stdout:rbd-mirror-18.0.0-5670.g74f8c0ae.el9.x86_64.rpm 6.2 MB/s | 3.0 MB 00:00 2023-08-24T10:04:18.929 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:04:18.930 INFO:teuthology.orchestra.run.smithi060.stdout:Total 6.1 MB/s | 3.0 MB 00:00 2023-08-24T10:04:18.930 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-08-24T10:04:18.940 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-08-24T10:04:18.941 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-08-24T10:04:19.021 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-08-24T10:04:19.022 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-08-24T10:04:19.464 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-08-24T10:04:19.546 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-mirror-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:19.572 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:19.573 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-24T10:04:19.573 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-08-24T10:04:19.573 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-08-24T10:04:19.573 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-08-24T10:04:19.573 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:20.719 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-mirror-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:20.720 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:20.720 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-08-24T10:04:20.720 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:04:20.720 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:20.721 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:04:20.983 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install rbd-nbd 2023-08-24T10:04:21.525 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:02:47 ago on Thu 24 Aug 2023 10:01:34 AM UTC. 2023-08-24T10:04:21.663 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-08-24T10:04:21.664 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:21.665 INFO:teuthology.orchestra.run.smithi060.stdout: Package Architecture Version Repository Size 2023-08-24T10:04:21.665 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:21.666 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-08-24T10:04:21.666 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd x86_64 2:18.0.0-5670.g74f8c0ae.el9 ceph 168 k 2023-08-24T10:04:21.666 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:21.666 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-08-24T10:04:21.667 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-08-24T10:04:21.667 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2023-08-24T10:04:21.667 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:21.667 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 168 k 2023-08-24T10:04:21.668 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 533 k 2023-08-24T10:04:21.668 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-08-24T10:04:21.886 INFO:teuthology.orchestra.run.smithi060.stdout:rbd-nbd-18.0.0-5670.g74f8c0ae.el9.x86_64.rpm 762 kB/s | 168 kB 00:00 2023-08-24T10:04:21.887 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-08-24T10:04:21.888 INFO:teuthology.orchestra.run.smithi060.stdout:Total 756 kB/s | 168 kB 00:00 2023-08-24T10:04:21.888 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-08-24T10:04:21.901 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-08-24T10:04:21.901 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-08-24T10:04:21.964 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-08-24T10:04:21.964 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-08-24T10:04:22.132 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-08-24T10:04:22.233 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-nbd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:22.727 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-nbd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:23.053 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-nbd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 1/1 2023-08-24T10:04:23.054 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:23.054 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-08-24T10:04:23.054 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd-2:18.0.0-5670.g74f8c0ae.el9.x86_64 2023-08-24T10:04:23.055 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:23.055 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-08-24T10:04:23.325 DEBUG:teuthology.parallel:result is None 2023-08-24T10:04:23.327 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe 2023-08-24T10:04:23.469 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-24T10:04:23.490 INFO:teuthology.orchestra.run.smithi060.stdout:18.0.0-5670.g74f8c0ae.el9 2023-08-24T10:04:23.491 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5670.g74f8c0ae.el9 2023-08-24T10:04:23.491 INFO:teuthology.task.install:The correct ceph version 18.0.0-5670.g74f8c0ae is installed. 2023-08-24T10:04:23.492 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe 2023-08-24T10:04:23.703 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-24T10:04:23.728 INFO:teuthology.orchestra.run.smithi135.stdout:18.0.0-5670.g74f8c0ae.el9 2023-08-24T10:04:23.728 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5670.g74f8c0ae.el9 2023-08-24T10:04:23.729 INFO:teuthology.task.install:The correct ceph version 18.0.0-5670.g74f8c0ae is installed. 2023-08-24T10:04:23.729 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-08-24T10:04:23.730 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-08-24T10:04:23.730 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-24T10:04:23.764 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-24T10:04:23.764 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-24T10:04:23.799 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-08-24T10:04:23.799 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-08-24T10:04:23.799 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/daemon-helper 2023-08-24T10:04:23.835 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-24T10:04:23.905 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-24T10:04:23.906 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/daemon-helper 2023-08-24T10:04:23.935 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-24T10:04:24.005 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-08-24T10:04:24.006 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-08-24T10:04:24.006 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-24T10:04:24.038 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-24T10:04:24.109 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-24T10:04:24.109 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-24T10:04:24.137 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-24T10:04:24.205 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-08-24T10:04:24.206 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-08-24T10:04:24.206 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/stdin-killer 2023-08-24T10:04:24.237 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-24T10:04:24.307 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-24T10:04:24.308 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/stdin-killer 2023-08-24T10:04:24.336 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-24T10:04:24.402 INFO:teuthology.run_tasks:Running task ceph... 2023-08-24T10:04:24.502 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-08-24T10:04:24.503 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 777 /var/log/ceph 2023-08-24T10:04:24.505 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 777 /var/log/ceph 2023-08-24T10:04:24.533 INFO:tasks.ceph:Disabling ceph logrotate... 2023-08-24T10:04:24.534 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/logrotate.d/ceph 2023-08-24T10:04:24.577 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /etc/logrotate.d/ceph 2023-08-24T10:04:24.604 INFO:tasks.ceph:Creating extra log directories... 2023-08-24T10:04:24.604 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-08-24T10:04:24.646 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-08-24T10:04:24.701 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-08-24T10:04:24.701 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson seastar smp': 3, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-08-24T10:04:24.701 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi/7378498', 'branch': 'wip-matanb-crimson-testing-24.8-v2', 'description': 'crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests}', 'email': None, 'first_in_suite': False, 'job_id': '7378498', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-matanb-crimson-testing-24.8-v2'}, 'ceph': {'conf': {'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'crimson seastar smp': 3, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'crimson', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe'}}, 'workunit': {'branch': 'wip-matanb-crimson-testing-24.8-v2', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe'}}, 'owner': 'scheduled_matan@teuthology', 'priority': 60, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': '74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'wip-matanb-crimson-testing-24.8-v2', 'suite_path': '/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/Matan-B/ceph.git', 'suite_sha1': '74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe', 'targets': {'smithi060.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN6gNUKYLjbh09BzRantQjCUQIPym0AcypVmYdTq0EQhIUlhNLQG2LbakWKji0BnsgZy+jdMt47mxajWjPvBTFc=', 'smithi135.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLiGtAHJjeXgDa3WaBv86LmWFjgoBL171vZvf/ibHaA9UP1RgxslYuGqz4owpEQtVYvE+83r0dcAeW86D74iIBc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson seastar smp': 3, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'flavor': 'crimson', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['rados/test.sh', 'rados/test_pool_quota.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '449a1bc2027504e7b3c3d7b30fa4178906581da7', 'timestamp': '2023-08-24_09:25:19', 'tube': 'smithi', 'user': 'matan', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2224'} 2023-08-24T10:04:24.702 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-08-24T10:04:24.716 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-08-24T10:04:24.730 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m0777 -- /var/run/ceph 2023-08-24T10:04:24.771 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m0777 -- /var/run/ceph 2023-08-24T10:04:24.799 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-08-24T10:04:24.799 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/scratch_devs of=/dev/stdout 2023-08-24T10:04:24.854 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-24T10:04:24.854 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_1 2023-08-24T10:04:24.909 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-24T10:04:24.909 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-24T10:04:24.909 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 654 Links: 1 2023-08-24T10:04:24.909 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-24T10:04:24.909 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-08-24T10:04:24.910 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-08-24 10:04:19.803688670 +0000 2023-08-24T10:04:24.910 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-08-24 10:00:39.421632808 +0000 2023-08-24T10:04:24.910 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-08-24 10:00:39.421632808 +0000 2023-08-24T10:04:24.910 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2023-08-24 10:00:39.421632808 +0000 2023-08-24T10:04:24.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-24T10:04:24.977 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-08-24T10:04:24.978 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-08-24T10:04:24.978 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000102726 s, 5.0 MB/s 2023-08-24T10:04:24.979 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-24T10:04:25.036 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_2 2023-08-24T10:04:25.092 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-24T10:04:25.093 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-24T10:04:25.093 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 671 Links: 1 2023-08-24T10:04:25.093 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-24T10:04:25.093 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-08-24T10:04:25.093 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-08-24 10:04:19.804688662 +0000 2023-08-24T10:04:25.093 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-08-24 10:00:39.939628957 +0000 2023-08-24T10:04:25.094 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-08-24 10:00:39.939628957 +0000 2023-08-24T10:04:25.094 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2023-08-24 10:00:39.939628957 +0000 2023-08-24T10:04:25.094 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-24T10:04:25.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-08-24T10:04:25.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-08-24T10:04:25.161 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 8.0379e-05 s, 6.4 MB/s 2023-08-24T10:04:25.162 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-24T10:04:25.218 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_3 2023-08-24T10:04:25.274 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-24T10:04:25.274 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-24T10:04:25.274 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 690 Links: 1 2023-08-24T10:04:25.274 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-24T10:04:25.275 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-08-24T10:04:25.275 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-08-24 10:04:19.804688662 +0000 2023-08-24T10:04:25.275 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-08-24 10:00:40.441625225 +0000 2023-08-24T10:04:25.275 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-08-24 10:00:40.441625225 +0000 2023-08-24T10:04:25.275 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2023-08-24 10:00:40.441625225 +0000 2023-08-24T10:04:25.275 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-24T10:04:25.343 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-08-24T10:04:25.343 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-08-24T10:04:25.344 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 7.8401e-05 s, 6.5 MB/s 2023-08-24T10:04:25.345 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-24T10:04:25.401 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_4 2023-08-24T10:04:25.456 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-24T10:04:25.456 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-24T10:04:25.456 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 710 Links: 1 2023-08-24T10:04:25.456 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-24T10:04:25.456 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-08-24T10:04:25.456 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-08-24 10:04:19.804688662 +0000 2023-08-24T10:04:25.457 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-08-24 10:00:40.937621538 +0000 2023-08-24T10:04:25.457 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-08-24 10:00:40.937621538 +0000 2023-08-24T10:04:25.457 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2023-08-24 10:00:40.937621538 +0000 2023-08-24T10:04:25.457 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-24T10:04:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-08-24T10:04:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-08-24T10:04:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 8.3032e-05 s, 6.2 MB/s 2023-08-24T10:04:25.526 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-24T10:04:25.582 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2023-08-24T10:04:25.583 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-24T10:04:25.583 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/scratch_devs of=/dev/stdout 2023-08-24T10:04:25.600 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-24T10:04:25.601 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_1 2023-08-24T10:04:25.656 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-24T10:04:25.657 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-24T10:04:25.657 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 653 Links: 1 2023-08-24T10:04:25.657 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-24T10:04:25.657 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2023-08-24T10:04:25.657 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2023-08-24 10:04:11.833859255 +0000 2023-08-24T10:04:25.657 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2023-08-24 10:00:25.684657793 +0000 2023-08-24T10:04:25.657 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2023-08-24 10:00:25.684657793 +0000 2023-08-24T10:04:25.658 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2023-08-24 10:00:25.684657793 +0000 2023-08-24T10:04:25.658 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-24T10:04:25.723 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2023-08-24T10:04:25.724 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2023-08-24T10:04:25.724 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000187889 s, 2.7 MB/s 2023-08-24T10:04:25.728 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-24T10:04:25.784 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_2 2023-08-24T10:04:25.844 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-24T10:04:25.844 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-24T10:04:25.844 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 673 Links: 1 2023-08-24T10:04:25.845 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-24T10:04:25.845 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2023-08-24T10:04:25.845 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2023-08-24 10:04:11.834859235 +0000 2023-08-24T10:04:25.845 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2023-08-24 10:00:26.195647396 +0000 2023-08-24T10:04:25.845 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2023-08-24 10:00:26.195647396 +0000 2023-08-24T10:04:25.845 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2023-08-24 10:00:26.195647396 +0000 2023-08-24T10:04:25.846 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-24T10:04:25.909 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2023-08-24T10:04:25.909 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2023-08-24T10:04:25.909 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000160263 s, 3.2 MB/s 2023-08-24T10:04:25.910 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-24T10:04:25.965 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_3 2023-08-24T10:04:26.020 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-24T10:04:26.020 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-24T10:04:26.020 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 691 Links: 1 2023-08-24T10:04:26.021 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-24T10:04:26.021 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2023-08-24T10:04:26.021 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2023-08-24 10:04:11.834859235 +0000 2023-08-24T10:04:26.021 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2023-08-24 10:00:26.674637650 +0000 2023-08-24T10:04:26.021 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2023-08-24 10:00:26.674637650 +0000 2023-08-24T10:04:26.021 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2023-08-24 10:00:26.674637650 +0000 2023-08-24T10:04:26.021 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-24T10:04:26.085 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2023-08-24T10:04:26.085 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2023-08-24T10:04:26.085 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000214516 s, 2.4 MB/s 2023-08-24T10:04:26.086 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-24T10:04:26.141 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_4 2023-08-24T10:04:26.197 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-24T10:04:26.197 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-24T10:04:26.197 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 709 Links: 1 2023-08-24T10:04:26.198 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-24T10:04:26.198 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2023-08-24T10:04:26.198 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2023-08-24 10:04:11.834859235 +0000 2023-08-24T10:04:26.199 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2023-08-24 10:00:27.178627395 +0000 2023-08-24T10:04:26.200 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2023-08-24 10:00:27.178627395 +0000 2023-08-24T10:04:26.200 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2023-08-24 10:00:27.178627395 +0000 2023-08-24T10:04:26.201 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-24T10:04:26.262 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2023-08-24T10:04:26.262 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2023-08-24T10:04:26.262 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000181596 s, 2.8 MB/s 2023-08-24T10:04:26.264 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-24T10:04:26.320 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2023-08-24T10:04:26.320 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi060.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi135.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2023-08-24T10:04:26.320 INFO:tasks.ceph:Generating config... 2023-08-24T10:04:26.321 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-08-24T10:04:26.322 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-08-24T10:04:26.322 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-08-24T10:04:26.322 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-08-24T10:04:26.322 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-08-24T10:04:26.323 INFO:tasks.ceph:[mon] debug mon = 20 2023-08-24T10:04:26.323 INFO:tasks.ceph:[mon] debug ms = 1 2023-08-24T10:04:26.323 INFO:tasks.ceph:[mon] debug paxos = 20 2023-08-24T10:04:26.323 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2023-08-24T10:04:26.324 INFO:tasks.ceph:[mon] osd pool default crimson = True 2023-08-24T10:04:26.324 INFO:tasks.ceph:[osd] debug monc = 20 2023-08-24T10:04:26.324 INFO:tasks.ceph:[osd] crimson seastar smp = 3 2023-08-24T10:04:26.324 INFO:tasks.ceph:[osd] debug ms = 1 2023-08-24T10:04:26.324 INFO:tasks.ceph:[osd] debug osd = 20 2023-08-24T10:04:26.325 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2023-08-24T10:04:26.325 INFO:tasks.ceph:[osd] osd class default list = * 2023-08-24T10:04:26.325 INFO:tasks.ceph:[osd] osd class load list = * 2023-08-24T10:04:26.325 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-08-24T10:04:26.325 INFO:tasks.ceph:[client] debug ms = 1 2023-08-24T10:04:26.326 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2023-08-24T10:04:26.326 INFO:tasks.ceph:[global] ms client mode = crc 2023-08-24T10:04:26.326 INFO:tasks.ceph:[global] ms cluster mode = crc 2023-08-24T10:04:26.326 INFO:tasks.ceph:[global] ms mon client mode = crc 2023-08-24T10:04:26.326 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2023-08-24T10:04:26.327 INFO:tasks.ceph:[global] ms mon service mode = crc 2023-08-24T10:04:26.327 INFO:tasks.ceph:[global] ms service mode = crc 2023-08-24T10:04:26.327 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-08-24T10:04:26.328 INFO:tasks.ceph:[mgr] debug ms = 1 2023-08-24T10:04:26.328 INFO:tasks.ceph:Setting up mon.a... 2023-08-24T10:04:26.328 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-08-24T10:04:26.406 INFO:teuthology.orchestra.run.smithi060.stdout:creating /etc/ceph/ceph.keyring 2023-08-24T10:04:26.409 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-08-24T10:04:26.464 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-24T10:04:26.495 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.60')] 2023-08-24T10:04:26.496 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.60', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson seastar smp': 3, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug ms': 1}, 'mon.a': {}} 2023-08-24T10:04:26.497 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-08-24T10:04:26.498 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-08-24T10:04:26.551 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.60 --print /home/ubuntu/cephtest/ceph.monmap 2023-08-24T10:04:26.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:26.635+0000 7fbe2780de80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:26.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:26.636+0000 7fbe2780de80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:26.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:26.636+0000 7fbe2780de80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:26.638 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-08-24T10:04:26.638 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: generated fsid bf8addf4-f373-46ca-83a9-ccdf9773802d 2023-08-24T10:04:26.638 INFO:teuthology.orchestra.run.smithi060.stdout:setting min_mon_release = pacific 2023-08-24T10:04:26.638 INFO:teuthology.orchestra.run.smithi060.stdout:epoch 0 2023-08-24T10:04:26.638 INFO:teuthology.orchestra.run.smithi060.stdout:fsid bf8addf4-f373-46ca-83a9-ccdf9773802d 2023-08-24T10:04:26.639 INFO:teuthology.orchestra.run.smithi060.stdout:last_changed 2023-08-24T10:04:26.637133+0000 2023-08-24T10:04:26.639 INFO:teuthology.orchestra.run.smithi060.stdout:created 2023-08-24T10:04:26.637133+0000 2023-08-24T10:04:26.639 INFO:teuthology.orchestra.run.smithi060.stdout:min_mon_release 16 (pacific) 2023-08-24T10:04:26.639 INFO:teuthology.orchestra.run.smithi060.stdout:election_strategy: 1 2023-08-24T10:04:26.639 INFO:teuthology.orchestra.run.smithi060.stdout:0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2023-08-24T10:04:26.639 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2023-08-24T10:04:26.640 DEBUG:teuthology.orchestra.run.smithi060:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-08-24T10:04:26.657 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID bf8addf4-f373-46ca-83a9-ccdf9773802d... 2023-08-24T10:04:26.660 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-08-24T10:04:26.700 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-08-24T10:04:26.751 INFO:teuthology.orchestra.run.smithi135.stdout:[global] 2023-08-24T10:04:26.751 INFO:teuthology.orchestra.run.smithi135.stdout: chdir = "" 2023-08-24T10:04:26.752 INFO:teuthology.orchestra.run.smithi135.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-08-24T10:04:26.752 INFO:teuthology.orchestra.run.smithi135.stdout: auth supported = cephx 2023-08-24T10:04:26.752 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.753 INFO:teuthology.orchestra.run.smithi135.stdout: filestore xattr use omap = true 2023-08-24T10:04:26.753 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.753 INFO:teuthology.orchestra.run.smithi135.stdout: mon clock drift allowed = 1.000 2023-08-24T10:04:26.753 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.754 INFO:teuthology.orchestra.run.smithi135.stdout: osd crush chooseleaf type = 0 2023-08-24T10:04:26.754 INFO:teuthology.orchestra.run.smithi135.stdout: auth debug = true 2023-08-24T10:04:26.754 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.754 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on old message = true 2023-08-24T10:04:26.754 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on bug = true 2023-08-24T10:04:26.755 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.755 INFO:teuthology.orchestra.run.smithi135.stdout: mon max pg per osd = 10000 # >= luminous 2023-08-24T10:04:26.755 INFO:teuthology.orchestra.run.smithi135.stdout: mon pg warn max object skew = 0 2023-08-24T10:04:26.756 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.757 INFO:teuthology.orchestra.run.smithi135.stdout: # disable pg_autoscaler by default for new pools 2023-08-24T10:04:26.757 INFO:teuthology.orchestra.run.smithi135.stdout: osd_pool_default_pg_autoscale_mode = off 2023-08-24T10:04:26.757 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.757 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default size = 2 2023-08-24T10:04:26.757 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.757 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow primary affinity = true 2023-08-24T10:04:26.757 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow pg remap = true 2023-08-24T10:04:26.757 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on legacy crush tunables = false 2023-08-24T10:04:26.758 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on crush straw calc version zero = false 2023-08-24T10:04:26.758 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on no sortbitwise = false 2023-08-24T10:04:26.758 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on osd down out interval zero = false 2023-08-24T10:04:26.758 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on too few osds = false 2023-08-24T10:04:26.758 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-08-24T10:04:26.758 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_no_redundancy = false 2023-08-24T10:04:26.759 INFO:teuthology.orchestra.run.smithi135.stdout: mon_allow_pool_size_one = true 2023-08-24T10:04:26.759 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.759 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-08-24T10:04:26.759 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.759 INFO:teuthology.orchestra.run.smithi135.stdout: osd default data pool replay window = 5 2023-08-24T10:04:26.759 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.760 INFO:teuthology.orchestra.run.smithi135.stdout: mon allow pool delete = true 2023-08-24T10:04:26.760 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.760 INFO:teuthology.orchestra.run.smithi135.stdout: mon cluster log file level = debug 2023-08-24T10:04:26.760 INFO:teuthology.orchestra.run.smithi135.stdout: debug asserts on shutdown = true 2023-08-24T10:04:26.760 INFO:teuthology.orchestra.run.smithi135.stdout: mon health detail to clog = false 2023-08-24T10:04:26.760 INFO:teuthology.orchestra.run.smithi135.stdout: mon host = 172.21.15.60 2023-08-24T10:04:26.760 INFO:teuthology.orchestra.run.smithi135.stdout: enable experimental unrecoverable data corrupting features = crimson 2023-08-24T10:04:26.761 INFO:teuthology.orchestra.run.smithi135.stdout: ms client mode = crc 2023-08-24T10:04:26.761 INFO:teuthology.orchestra.run.smithi135.stdout: ms cluster mode = crc 2023-08-24T10:04:26.761 INFO:teuthology.orchestra.run.smithi135.stdout: ms mon client mode = crc 2023-08-24T10:04:26.761 INFO:teuthology.orchestra.run.smithi135.stdout: ms mon cluster mode = crc 2023-08-24T10:04:26.761 INFO:teuthology.orchestra.run.smithi135.stdout: ms mon service mode = crc 2023-08-24T10:04:26.761 INFO:teuthology.orchestra.run.smithi135.stdout: ms service mode = crc 2023-08-24T10:04:26.761 INFO:teuthology.orchestra.run.smithi135.stdout: fsid = bf8addf4-f373-46ca-83a9-ccdf9773802d 2023-08-24T10:04:26.762 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.762 INFO:teuthology.orchestra.run.smithi135.stdout:[osd] 2023-08-24T10:04:26.762 INFO:teuthology.orchestra.run.smithi135.stdout: osd journal size = 100 2023-08-24T10:04:26.762 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.762 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub load threshold = 5.0 2023-08-24T10:04:26.763 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub max interval = 600 2023-08-24T10:04:26.763 INFO:teuthology.orchestra.run.smithi135.stdout: osd mclock profile = high_recovery_ops 2023-08-24T10:04:26.763 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.763 INFO:teuthology.orchestra.run.smithi135.stdout: osd recover clone overlap = true 2023-08-24T10:04:26.763 INFO:teuthology.orchestra.run.smithi135.stdout: osd recovery max chunk = 1048576 2023-08-24T10:04:26.763 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.763 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug shutdown = true 2023-08-24T10:04:26.764 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug op order = true 2023-08-24T10:04:26.764 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug verify stray on activate = true 2023-08-24T10:04:26.764 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.764 INFO:teuthology.orchestra.run.smithi135.stdout: osd open classes on start = true 2023-08-24T10:04:26.764 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug pg log writeout = true 2023-08-24T10:04:26.764 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.764 INFO:teuthology.orchestra.run.smithi135.stdout: osd deep scrub update digest min age = 30 2023-08-24T10:04:26.764 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.765 INFO:teuthology.orchestra.run.smithi135.stdout: osd map max advance = 10 2023-08-24T10:04:26.765 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.765 INFO:teuthology.orchestra.run.smithi135.stdout: journal zero on create = true 2023-08-24T10:04:26.765 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.765 INFO:teuthology.orchestra.run.smithi135.stdout: filestore ondisk finisher threads = 3 2023-08-24T10:04:26.765 INFO:teuthology.orchestra.run.smithi135.stdout: filestore apply finisher threads = 3 2023-08-24T10:04:26.765 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.766 INFO:teuthology.orchestra.run.smithi135.stdout: bdev debug aio = true 2023-08-24T10:04:26.766 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug misdirected ops = true 2023-08-24T10:04:26.766 INFO:teuthology.orchestra.run.smithi135.stdout: debug monc = 20 2023-08-24T10:04:26.767 INFO:teuthology.orchestra.run.smithi135.stdout: crimson seastar smp = 3 2023-08-24T10:04:26.767 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2023-08-24T10:04:26.767 INFO:teuthology.orchestra.run.smithi135.stdout: debug osd = 20 2023-08-24T10:04:26.767 INFO:teuthology.orchestra.run.smithi135.stdout: osd blocked scrub grace period = 3600 2023-08-24T10:04:26.767 INFO:teuthology.orchestra.run.smithi135.stdout: osd class default list = * 2023-08-24T10:04:26.767 INFO:teuthology.orchestra.run.smithi135.stdout: osd class load list = * 2023-08-24T10:04:26.767 INFO:teuthology.orchestra.run.smithi135.stdout: osd shutdown pgref assert = True 2023-08-24T10:04:26.768 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.768 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr] 2023-08-24T10:04:26.768 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2023-08-24T10:04:26.768 INFO:teuthology.orchestra.run.smithi135.stdout: debug mgr = 20 2023-08-24T10:04:26.768 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2023-08-24T10:04:26.768 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2023-08-24T10:04:26.768 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2023-08-24T10:04:26.769 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2023-08-24T10:04:26.769 INFO:teuthology.orchestra.run.smithi135.stdout: mgr/telemetry/nag = false 2023-08-24T10:04:26.769 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.769 INFO:teuthology.orchestra.run.smithi135.stdout:[mon] 2023-08-24T10:04:26.769 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2023-08-24T10:04:26.769 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2023-08-24T10:04:26.769 INFO:teuthology.orchestra.run.smithi135.stdout: debug paxos = 20 2023-08-24T10:04:26.769 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2023-08-24T10:04:26.770 INFO:teuthology.orchestra.run.smithi135.stdout: mon data avail warn = 5 2023-08-24T10:04:26.770 INFO:teuthology.orchestra.run.smithi135.stdout: mon mgr mkfs grace = 240 2023-08-24T10:04:26.770 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2023-08-24T10:04:26.770 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd reporter subtree level = osd 2023-08-24T10:04:26.770 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd prime pg temp = true 2023-08-24T10:04:26.770 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2023-08-24T10:04:26.770 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.771 INFO:teuthology.orchestra.run.smithi135.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-08-24T10:04:26.771 INFO:teuthology.orchestra.run.smithi135.stdout: auth mon ticket ttl = 660 # 11m 2023-08-24T10:04:26.771 INFO:teuthology.orchestra.run.smithi135.stdout: auth service ticket ttl = 240 # 4m 2023-08-24T10:04:26.771 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.771 INFO:teuthology.orchestra.run.smithi135.stdout: # don't complain about insecure global_id in the test suite 2023-08-24T10:04:26.771 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-08-24T10:04:26.771 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-08-24T10:04:26.771 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.772 INFO:teuthology.orchestra.run.smithi135.stdout: # 1m isn't quite enough 2023-08-24T10:04:26.772 INFO:teuthology.orchestra.run.smithi135.stdout: mon_down_mkfs_grace = 2m 2023-08-24T10:04:26.772 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.772 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_filestore_osds = false 2023-08-24T10:04:26.772 INFO:teuthology.orchestra.run.smithi135.stdout: mon min osdmap epochs = 50 2023-08-24T10:04:26.772 INFO:teuthology.orchestra.run.smithi135.stdout: mon osdmap full prune interval = 2 2023-08-24T10:04:26.772 INFO:teuthology.orchestra.run.smithi135.stdout: mon osdmap full prune min = 15 2023-08-24T10:04:26.773 INFO:teuthology.orchestra.run.smithi135.stdout: mon osdmap full prune txsize = 2 2023-08-24T10:04:26.773 INFO:teuthology.orchestra.run.smithi135.stdout: paxos service trim min = 10 2023-08-24T10:04:26.773 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on pool no app = False 2023-08-24T10:04:26.773 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default crimson = True 2023-08-24T10:04:26.773 INFO:teuthology.orchestra.run.smithi135.stdout: 2023-08-24T10:04:26.773 INFO:teuthology.orchestra.run.smithi135.stdout:[client] 2023-08-24T10:04:26.773 INFO:teuthology.orchestra.run.smithi135.stdout: rgw cache enabled = true 2023-08-24T10:04:26.774 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable ops log = true 2023-08-24T10:04:26.774 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable usage log = true 2023-08-24T10:04:26.774 INFO:teuthology.orchestra.run.smithi135.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-08-24T10:04:26.774 INFO:teuthology.orchestra.run.smithi135.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-08-24T10:04:26.774 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2023-08-24T10:04:26.774 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.a] 2023-08-24T10:04:26.775 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2023-08-24T10:04:26.775 INFO:teuthology.orchestra.run.smithi060.stdout: chdir = "" 2023-08-24T10:04:26.775 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-08-24T10:04:26.775 INFO:teuthology.orchestra.run.smithi060.stdout: auth supported = cephx 2023-08-24T10:04:26.776 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.776 INFO:teuthology.orchestra.run.smithi060.stdout: filestore xattr use omap = true 2023-08-24T10:04:26.776 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.776 INFO:teuthology.orchestra.run.smithi060.stdout: mon clock drift allowed = 1.000 2023-08-24T10:04:26.776 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.776 INFO:teuthology.orchestra.run.smithi060.stdout: osd crush chooseleaf type = 0 2023-08-24T10:04:26.776 INFO:teuthology.orchestra.run.smithi060.stdout: auth debug = true 2023-08-24T10:04:26.776 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.777 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on old message = true 2023-08-24T10:04:26.777 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on bug = true 2023-08-24T10:04:26.777 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.777 INFO:teuthology.orchestra.run.smithi060.stdout: mon max pg per osd = 10000 # >= luminous 2023-08-24T10:04:26.777 INFO:teuthology.orchestra.run.smithi060.stdout: mon pg warn max object skew = 0 2023-08-24T10:04:26.777 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.777 INFO:teuthology.orchestra.run.smithi060.stdout: # disable pg_autoscaler by default for new pools 2023-08-24T10:04:26.778 INFO:teuthology.orchestra.run.smithi060.stdout: osd_pool_default_pg_autoscale_mode = off 2023-08-24T10:04:26.778 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.778 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default size = 2 2023-08-24T10:04:26.778 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.778 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow primary affinity = true 2023-08-24T10:04:26.778 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow pg remap = true 2023-08-24T10:04:26.778 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on legacy crush tunables = false 2023-08-24T10:04:26.778 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on crush straw calc version zero = false 2023-08-24T10:04:26.779 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on no sortbitwise = false 2023-08-24T10:04:26.779 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on osd down out interval zero = false 2023-08-24T10:04:26.779 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on too few osds = false 2023-08-24T10:04:26.779 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-08-24T10:04:26.779 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_no_redundancy = false 2023-08-24T10:04:26.780 INFO:teuthology.orchestra.run.smithi060.stdout: mon_allow_pool_size_one = true 2023-08-24T10:04:26.780 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.780 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-08-24T10:04:26.780 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.780 INFO:teuthology.orchestra.run.smithi060.stdout: osd default data pool replay window = 5 2023-08-24T10:04:26.780 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.780 INFO:teuthology.orchestra.run.smithi060.stdout: mon allow pool delete = true 2023-08-24T10:04:26.781 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.781 INFO:teuthology.orchestra.run.smithi060.stdout: mon cluster log file level = debug 2023-08-24T10:04:26.781 INFO:teuthology.orchestra.run.smithi060.stdout: debug asserts on shutdown = true 2023-08-24T10:04:26.781 INFO:teuthology.orchestra.run.smithi060.stdout: mon health detail to clog = false 2023-08-24T10:04:26.781 INFO:teuthology.orchestra.run.smithi060.stdout: mon host = 172.21.15.60 2023-08-24T10:04:26.781 INFO:teuthology.orchestra.run.smithi060.stdout: enable experimental unrecoverable data corrupting features = crimson 2023-08-24T10:04:26.781 INFO:teuthology.orchestra.run.smithi060.stdout: ms client mode = crc 2023-08-24T10:04:26.781 INFO:teuthology.orchestra.run.smithi060.stdout: ms cluster mode = crc 2023-08-24T10:04:26.782 INFO:teuthology.orchestra.run.smithi060.stdout: ms mon client mode = crc 2023-08-24T10:04:26.782 INFO:teuthology.orchestra.run.smithi060.stdout: ms mon cluster mode = crc 2023-08-24T10:04:26.782 INFO:teuthology.orchestra.run.smithi060.stdout: ms mon service mode = crc 2023-08-24T10:04:26.782 INFO:teuthology.orchestra.run.smithi060.stdout: ms service mode = crc 2023-08-24T10:04:26.782 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = bf8addf4-f373-46ca-83a9-ccdf9773802d 2023-08-24T10:04:26.782 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.782 INFO:teuthology.orchestra.run.smithi060.stdout:[osd] 2023-08-24T10:04:26.782 INFO:teuthology.orchestra.run.smithi060.stdout: osd journal size = 100 2023-08-24T10:04:26.783 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.783 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub load threshold = 5.0 2023-08-24T10:04:26.783 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub max interval = 600 2023-08-24T10:04:26.783 INFO:teuthology.orchestra.run.smithi060.stdout: osd mclock profile = high_recovery_ops 2023-08-24T10:04:26.783 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.783 INFO:teuthology.orchestra.run.smithi060.stdout: osd recover clone overlap = true 2023-08-24T10:04:26.783 INFO:teuthology.orchestra.run.smithi060.stdout: osd recovery max chunk = 1048576 2023-08-24T10:04:26.783 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.784 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug shutdown = true 2023-08-24T10:04:26.784 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug op order = true 2023-08-24T10:04:26.784 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug verify stray on activate = true 2023-08-24T10:04:26.784 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.784 INFO:teuthology.orchestra.run.smithi060.stdout: osd open classes on start = true 2023-08-24T10:04:26.785 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug pg log writeout = true 2023-08-24T10:04:26.785 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.785 INFO:teuthology.orchestra.run.smithi060.stdout: osd deep scrub update digest min age = 30 2023-08-24T10:04:26.785 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.785 INFO:teuthology.orchestra.run.smithi060.stdout: osd map max advance = 10 2023-08-24T10:04:26.785 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.786 INFO:teuthology.orchestra.run.smithi060.stdout: journal zero on create = true 2023-08-24T10:04:26.786 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.786 INFO:teuthology.orchestra.run.smithi060.stdout: filestore ondisk finisher threads = 3 2023-08-24T10:04:26.786 INFO:teuthology.orchestra.run.smithi060.stdout: filestore apply finisher threads = 3 2023-08-24T10:04:26.786 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.786 INFO:teuthology.orchestra.run.smithi060.stdout: bdev debug aio = true 2023-08-24T10:04:26.786 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug misdirected ops = true 2023-08-24T10:04:26.786 INFO:teuthology.orchestra.run.smithi060.stdout: debug monc = 20 2023-08-24T10:04:26.787 INFO:teuthology.orchestra.run.smithi060.stdout: crimson seastar smp = 3 2023-08-24T10:04:26.787 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-08-24T10:04:26.787 INFO:teuthology.orchestra.run.smithi060.stdout: debug osd = 20 2023-08-24T10:04:26.787 INFO:teuthology.orchestra.run.smithi060.stdout: osd blocked scrub grace period = 3600 2023-08-24T10:04:26.787 INFO:teuthology.orchestra.run.smithi060.stdout: osd class default list = * 2023-08-24T10:04:26.788 INFO:teuthology.orchestra.run.smithi060.stdout: osd class load list = * 2023-08-24T10:04:26.788 INFO:teuthology.orchestra.run.smithi060.stdout: osd shutdown pgref assert = True 2023-08-24T10:04:26.788 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.788 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr] 2023-08-24T10:04:26.788 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-08-24T10:04:26.788 INFO:teuthology.orchestra.run.smithi060.stdout: debug mgr = 20 2023-08-24T10:04:26.788 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2023-08-24T10:04:26.789 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2023-08-24T10:04:26.789 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2023-08-24T10:04:26.789 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2023-08-24T10:04:26.789 INFO:teuthology.orchestra.run.smithi060.stdout: mgr/telemetry/nag = false 2023-08-24T10:04:26.789 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.789 INFO:teuthology.orchestra.run.smithi060.stdout:[mon] 2023-08-24T10:04:26.789 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-08-24T10:04:26.789 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2023-08-24T10:04:26.790 INFO:teuthology.orchestra.run.smithi060.stdout: debug paxos = 20 2023-08-24T10:04:26.790 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2023-08-24T10:04:26.790 INFO:teuthology.orchestra.run.smithi060.stdout: mon data avail warn = 5 2023-08-24T10:04:26.790 INFO:teuthology.orchestra.run.smithi060.stdout: mon mgr mkfs grace = 240 2023-08-24T10:04:26.790 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2023-08-24T10:04:26.790 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd reporter subtree level = osd 2023-08-24T10:04:26.790 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd prime pg temp = true 2023-08-24T10:04:26.790 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2023-08-24T10:04:26.791 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.791 INFO:teuthology.orchestra.run.smithi060.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-08-24T10:04:26.791 INFO:teuthology.orchestra.run.smithi060.stdout: auth mon ticket ttl = 660 # 11m 2023-08-24T10:04:26.791 INFO:teuthology.orchestra.run.smithi060.stdout: auth service ticket ttl = 240 # 4m 2023-08-24T10:04:26.791 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.791 INFO:teuthology.orchestra.run.smithi060.stdout: # don't complain about insecure global_id in the test suite 2023-08-24T10:04:26.791 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-08-24T10:04:26.791 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-08-24T10:04:26.792 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.792 INFO:teuthology.orchestra.run.smithi060.stdout: # 1m isn't quite enough 2023-08-24T10:04:26.792 INFO:teuthology.orchestra.run.smithi060.stdout: mon_down_mkfs_grace = 2m 2023-08-24T10:04:26.792 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.792 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_filestore_osds = false 2023-08-24T10:04:26.792 INFO:teuthology.orchestra.run.smithi060.stdout: mon min osdmap epochs = 50 2023-08-24T10:04:26.792 INFO:teuthology.orchestra.run.smithi060.stdout: mon osdmap full prune interval = 2 2023-08-24T10:04:26.793 INFO:teuthology.orchestra.run.smithi060.stdout: mon osdmap full prune min = 15 2023-08-24T10:04:26.793 INFO:teuthology.orchestra.run.smithi060.stdout: mon osdmap full prune txsize = 2 2023-08-24T10:04:26.793 INFO:teuthology.orchestra.run.smithi060.stdout: paxos service trim min = 10 2023-08-24T10:04:26.793 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on pool no app = False 2023-08-24T10:04:26.793 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default crimson = True 2023-08-24T10:04:26.793 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:26.793 INFO:teuthology.orchestra.run.smithi060.stdout:[client] 2023-08-24T10:04:26.793 INFO:teuthology.orchestra.run.smithi060.stdout: rgw cache enabled = true 2023-08-24T10:04:26.794 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable ops log = true 2023-08-24T10:04:26.794 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable usage log = true 2023-08-24T10:04:26.794 INFO:teuthology.orchestra.run.smithi060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-08-24T10:04:26.794 INFO:teuthology.orchestra.run.smithi060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-08-24T10:04:26.794 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-08-24T10:04:26.794 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.a] 2023-08-24T10:04:26.795 INFO:tasks.ceph:Creating admin key on mon.a... 2023-08-24T10:04:26.796 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-08-24T10:04:26.902 INFO:tasks.ceph:Copying monmap to all nodes... 2023-08-24T10:04:26.903 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-08-24T10:04:26.903 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-08-24T10:04:26.918 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-08-24T10:04:26.918 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-08-24T10:04:26.973 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi060.front.sepia.ceph.com 2023-08-24T10:04:26.974 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-08-24T10:04:26.974 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.keyring 2023-08-24T10:04:26.974 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-24T10:04:27.059 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-08-24T10:04:27.060 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-08-24T10:04:27.115 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi135.front.sepia.ceph.com 2023-08-24T10:04:27.115 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-24T10:04:27.115 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.keyring 2023-08-24T10:04:27.116 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-24T10:04:27.161 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-24T10:04:27.161 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-08-24T10:04:27.215 INFO:tasks.ceph:Setting up mon nodes... 2023-08-24T10:04:27.215 INFO:tasks.ceph:Setting up mgr nodes... 2023-08-24T10:04:27.216 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-08-24T10:04:27.341 INFO:teuthology.orchestra.run.smithi135.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-08-24T10:04:27.345 INFO:tasks.ceph:Setting up mds nodes... 2023-08-24T10:04:27.345 INFO:tasks.ceph_client:Setting up client nodes... 2023-08-24T10:04:27.346 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-08-24T10:04:27.399 INFO:teuthology.orchestra.run.smithi060.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-08-24T10:04:27.417 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-08-24T10:04:27.465 INFO:teuthology.orchestra.run.smithi135.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-08-24T10:04:27.481 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-08-24T10:04:27.481 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi060.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi135.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2023-08-24T10:04:27.482 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-08-24T10:04:27.512 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2023-08-24T10:04:27.512 INFO:tasks.ceph:role: osd.0 2023-08-24T10:04:27.512 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi060.front.sepia.ceph.com 2023-08-24T10:04:27.513 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-08-24T10:04:27.581 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-08-24T10:04:27.582 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-24T10:04:27.582 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-24T10:04:27.582 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=1 inobtcount=1 2023-08-24T10:04:27.582 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-24T10:04:27.582 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2023-08-24T10:04:27.582 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-24T10:04:27.582 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-24T10:04:27.583 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-24T10:04:27.583 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-24T10:04:27.607 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2023-08-24T10:04:27.609 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2023-08-24T10:04:27.609 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-08-24T10:04:27.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-08-24T10:04:27.759 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-08-24T10:04:27.829 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2023-08-24T10:04:27.830 INFO:tasks.ceph:role: osd.1 2023-08-24T10:04:27.830 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi060.front.sepia.ceph.com 2023-08-24T10:04:27.830 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-08-24T10:04:27.900 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-08-24T10:04:27.900 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-24T10:04:27.900 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-24T10:04:27.900 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=1 inobtcount=1 2023-08-24T10:04:27.900 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-24T10:04:27.901 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2023-08-24T10:04:27.901 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-24T10:04:27.901 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-24T10:04:27.901 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-24T10:04:27.901 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-24T10:04:27.926 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2023-08-24T10:04:27.928 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2023-08-24T10:04:27.928 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-08-24T10:04:28.002 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-08-24T10:04:28.076 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-24T10:04:28.624 INFO:teuthology.orchestra.run.smithi060.stderr:WARNING: debug mode. Not for benchmarking or production 2023-08-24T10:04:28.624 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:28,624 seastar - Reactor backend: linux-aio 2023-08-24T10:04:28.641 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:28,640 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-08-24T10:04:28.641 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:28,640 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-08-24T10:04:28.641 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:28,640 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-08-24T10:04:28.642 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:28,640 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-08-24T10:04:28.643 INFO:teuthology.orchestra.run.smithi060.stderr:==81051==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-08-24T10:04:28.649 INFO:teuthology.orchestra.run.smithi060.stderr:==81051==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7fffa61ef000; bottom 0x7f12c56f3000; size: 0x00ece0afc000 (1017381896192) 2023-08-24T10:04:28.649 INFO:teuthology.orchestra.run.smithi060.stderr:False positive error reports may follow 2023-08-24T10:04:28.649 INFO:teuthology.orchestra.run.smithi060.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-08-24T10:04:28.685 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:28,685 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2023-08-24T10:04:28.685 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:28,685 [shard 0] osd - get_early_config: set --smp 3 2023-08-24T10:04:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:----------------------------------------------------- 2023-08-24T10:04:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:Suppressions used: 2023-08-24T10:04:29.194 INFO:teuthology.orchestra.run.smithi060.stderr: count bytes template 2023-08-24T10:04:29.194 INFO:teuthology.orchestra.run.smithi060.stderr: 1 8 MallocExtension::Register 2023-08-24T10:04:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:----------------------------------------------------- 2023-08-24T10:04:29.195 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-08-24T10:04:29.250 INFO:teuthology.orchestra.run.smithi060.stderr:WARNING: debug mode. Not for benchmarking or production 2023-08-24T10:04:29.251 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:29,250 seastar - Reactor backend: linux-aio 2023-08-24T10:04:29.268 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:29,267 [shard 1] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-08-24T10:04:29.268 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:29,268 [shard 1] seastar - IO queue uses 0.75ms latency goal for device 0 2023-08-24T10:04:29.268 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:29,268 [shard 1] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-08-24T10:04:29.275 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:29,275 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-08-24T10:04:29.281 INFO:teuthology.orchestra.run.smithi060.stderr:==81049==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-08-24T10:04:29.284 INFO:teuthology.orchestra.run.smithi060.stderr:==81049==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7fffa61ef000; bottom 0x7f12c34dc000; size: 0x00ece2d13000 (1017417641984) 2023-08-24T10:04:29.284 INFO:teuthology.orchestra.run.smithi060.stderr:False positive error reports may follow 2023-08-24T10:04:29.284 INFO:teuthology.orchestra.run.smithi060.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-08-24T10:04:29.367 INFO:teuthology.orchestra.run.smithi060.stderr:Reactor stalled for 33 ms on shard 0. Backtrace: 0x45d5d 0x2c041bb8 0x2c042f66 0x2c0444b4 0x2c044838 0x2c044982 0x2c044dd8 0x54def 0xce1ca 0x588c4 0xdbaa8 0x25d32e4c 0x25d35495 0x25d372a6 0x25d39b0b 0x25a0586f 0x1e8d65f6 0x1e84832b 0x1e84c297 0x2c3c581f 2023-08-24T10:04:29.368 INFO:teuthology.orchestra.run.smithi060.stderr:kernel callstack: 2023-08-24T10:04:32.889 INFO:teuthology.orchestra.run.smithi060.stderr:----------------------------------------------------- 2023-08-24T10:04:32.889 INFO:teuthology.orchestra.run.smithi060.stderr:Suppressions used: 2023-08-24T10:04:32.890 INFO:teuthology.orchestra.run.smithi060.stderr: count bytes template 2023-08-24T10:04:32.890 INFO:teuthology.orchestra.run.smithi060.stderr: 1 8 MallocExtension::Register 2023-08-24T10:04:32.890 INFO:teuthology.orchestra.run.smithi060.stderr:----------------------------------------------------- 2023-08-24T10:04:32.890 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-08-24T10:04:32.890 INFO:teuthology.orchestra.run.smithi060.stdout:created object store /var/lib/ceph/osd/ceph-0 for osd.0 fsid bf8addf4-f373-46ca-83a9-ccdf9773802d 2023-08-24T10:04:32.929 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-08-24T10:04:32.960 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-24T10:04:33.499 INFO:teuthology.orchestra.run.smithi060.stderr:WARNING: debug mode. Not for benchmarking or production 2023-08-24T10:04:33.499 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:33,499 seastar - Reactor backend: linux-aio 2023-08-24T10:04:33.516 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:33,515 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-08-24T10:04:33.517 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:33,515 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-08-24T10:04:33.517 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:33,515 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-08-24T10:04:33.517 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:33,515 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-08-24T10:04:33.518 INFO:teuthology.orchestra.run.smithi060.stderr:==81859==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-08-24T10:04:33.525 INFO:teuthology.orchestra.run.smithi060.stderr:==81859==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffe60342000; bottom 0x7fdd7a0e3000; size: 0x0020e625f000 (141300199424) 2023-08-24T10:04:33.525 INFO:teuthology.orchestra.run.smithi060.stderr:False positive error reports may follow 2023-08-24T10:04:33.525 INFO:teuthology.orchestra.run.smithi060.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-08-24T10:04:33.550 INFO:teuthology.orchestra.run.smithi060.stderr:Reactor stalled for 32 ms on shard 0. Backtrace: 0x45d5d 0x2c041bb8 0x2c042f66 0x2c0444b4 0x2c044838 0x2c044982 0x2c044dd8 0x54def 0x28821 0xb738c 0x228ff04e 0x225acfbe 0x225daa11 0x2457b2f1 0x24580a19 0x24582fb4 0x2458418d 0x22167a9f 0x221bb388 0x221d9ab3 0x2bfe3ff9 0x2c081cee 0x2c2a5af4 0x2c2a793f 0x2bd002e6 0x2bd021e7 0x1e9927eb 0x1e99c959 0x1e818351 0x3feaf 0x3ff5f 0x1e2d3a04 2023-08-24T10:04:33.550 INFO:teuthology.orchestra.run.smithi060.stderr:kernel callstack: 2023-08-24T10:04:33.560 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:33,559 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2023-08-24T10:04:33.560 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:33,559 [shard 0] osd - get_early_config: set --smp 3 2023-08-24T10:04:34.052 INFO:teuthology.orchestra.run.smithi060.stderr:----------------------------------------------------- 2023-08-24T10:04:34.052 INFO:teuthology.orchestra.run.smithi060.stderr:Suppressions used: 2023-08-24T10:04:34.052 INFO:teuthology.orchestra.run.smithi060.stderr: count bytes template 2023-08-24T10:04:34.053 INFO:teuthology.orchestra.run.smithi060.stderr: 1 8 MallocExtension::Register 2023-08-24T10:04:34.053 INFO:teuthology.orchestra.run.smithi060.stderr:----------------------------------------------------- 2023-08-24T10:04:34.053 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-08-24T10:04:34.114 INFO:teuthology.orchestra.run.smithi060.stderr:WARNING: debug mode. Not for benchmarking or production 2023-08-24T10:04:34.115 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:34,115 seastar - Reactor backend: linux-aio 2023-08-24T10:04:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:34,133 [shard 1] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-08-24T10:04:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:34,133 [shard 1] seastar - IO queue uses 0.75ms latency goal for device 0 2023-08-24T10:04:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:34,133 [shard 1] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-08-24T10:04:34.141 INFO:teuthology.orchestra.run.smithi060.stderr:INFO 2023-08-24 10:04:34,140 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-08-24T10:04:34.143 INFO:teuthology.orchestra.run.smithi060.stderr:==81857==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-08-24T10:04:34.149 INFO:teuthology.orchestra.run.smithi060.stderr:==81857==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffe60342000; bottom 0x7fdd77ecc000; size: 0x0020e8476000 (141335945216) 2023-08-24T10:04:34.150 INFO:teuthology.orchestra.run.smithi060.stderr:False positive error reports may follow 2023-08-24T10:04:34.150 INFO:teuthology.orchestra.run.smithi060.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-08-24T10:04:34.232 INFO:teuthology.orchestra.run.smithi060.stderr:Reactor stalled for 32 ms on shard 0. Backtrace: 0x45d5d 0x2c041bb8 0x2c042f66 0x2c0444b4 0x2c044838 0x2c044982 0x2c044dd8 0x54def 0xce1ca 0x588c4 0xdbaa8 0x25d32e4c 0x25d35495 0x25d372a6 0x25d39b0b 0x25a0586f 0x1e8d65f6 0x1e84832b 0x1e84c297 0x2c3c581f 2023-08-24T10:04:34.232 INFO:teuthology.orchestra.run.smithi060.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff96c54847 0xffffffff96c50167 0xffffffff96e000aa 2023-08-24T10:04:37.741 INFO:teuthology.orchestra.run.smithi060.stderr:----------------------------------------------------- 2023-08-24T10:04:37.742 INFO:teuthology.orchestra.run.smithi060.stderr:Suppressions used: 2023-08-24T10:04:37.742 INFO:teuthology.orchestra.run.smithi060.stderr: count bytes template 2023-08-24T10:04:37.742 INFO:teuthology.orchestra.run.smithi060.stderr: 1 8 MallocExtension::Register 2023-08-24T10:04:37.742 INFO:teuthology.orchestra.run.smithi060.stderr:----------------------------------------------------- 2023-08-24T10:04:37.742 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-08-24T10:04:37.743 INFO:teuthology.orchestra.run.smithi060.stdout:created object store /var/lib/ceph/osd/ceph-1 for osd.1 fsid bf8addf4-f373-46ca-83a9-ccdf9773802d 2023-08-24T10:04:37.795 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-08-24T10:04:37.825 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-08-24T10:04:37.858 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2023-08-24T10:04:37.858 INFO:tasks.ceph:role: osd.2 2023-08-24T10:04:37.859 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi135.front.sepia.ceph.com 2023-08-24T10:04:37.859 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-08-24T10:04:37.936 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-08-24T10:04:37.936 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-24T10:04:37.936 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-24T10:04:37.936 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=1 inobtcount=1 2023-08-24T10:04:37.936 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-24T10:04:37.936 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2023-08-24T10:04:37.937 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-24T10:04:37.937 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-24T10:04:37.937 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-24T10:04:37.937 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-24T10:04:38.404 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2023-08-24T10:04:38.404 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2023-08-24T10:04:38.405 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2023-08-24T10:04:38.550 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-08-24T10:04:38.586 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-08-24T10:04:38.655 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2023-08-24T10:04:38.655 INFO:tasks.ceph:role: osd.3 2023-08-24T10:04:38.656 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi135.front.sepia.ceph.com 2023-08-24T10:04:38.656 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-08-24T10:04:38.731 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-08-24T10:04:38.731 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-24T10:04:38.732 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-24T10:04:38.732 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=1 inobtcount=1 2023-08-24T10:04:38.732 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-24T10:04:38.733 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2023-08-24T10:04:38.733 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-24T10:04:38.734 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-24T10:04:38.734 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-24T10:04:38.734 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-24T10:04:39.169 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2023-08-24T10:04:39.170 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2023-08-24T10:04:39.172 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2023-08-24T10:04:39.270 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-08-24T10:04:39.304 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-24T10:04:39.819 INFO:teuthology.orchestra.run.smithi135.stderr:WARNING: debug mode. Not for benchmarking or production 2023-08-24T10:04:39.820 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:39,819 seastar - Reactor backend: linux-aio 2023-08-24T10:04:39.838 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:39,837 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-08-24T10:04:39.838 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:39,837 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-08-24T10:04:39.838 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:39,837 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-08-24T10:04:39.838 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:39,837 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-08-24T10:04:39.840 INFO:teuthology.orchestra.run.smithi135.stderr:==80915==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-08-24T10:04:39.846 INFO:teuthology.orchestra.run.smithi135.stderr:==80915==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7fff4cc00000; bottom 0x7fb3b7595000; size: 0x004b9566b000 (324629082112) 2023-08-24T10:04:39.846 INFO:teuthology.orchestra.run.smithi135.stderr:False positive error reports may follow 2023-08-24T10:04:39.846 INFO:teuthology.orchestra.run.smithi135.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-08-24T10:04:39.879 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:39,879 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2023-08-24T10:04:39.880 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:39,879 [shard 0] osd - get_early_config: set --smp 3 2023-08-24T10:04:40.362 INFO:teuthology.orchestra.run.smithi135.stderr:----------------------------------------------------- 2023-08-24T10:04:40.362 INFO:teuthology.orchestra.run.smithi135.stderr:Suppressions used: 2023-08-24T10:04:40.362 INFO:teuthology.orchestra.run.smithi135.stderr: count bytes template 2023-08-24T10:04:40.363 INFO:teuthology.orchestra.run.smithi135.stderr: 1 8 MallocExtension::Register 2023-08-24T10:04:40.363 INFO:teuthology.orchestra.run.smithi135.stderr:----------------------------------------------------- 2023-08-24T10:04:40.363 INFO:teuthology.orchestra.run.smithi135.stderr: 2023-08-24T10:04:40.413 INFO:teuthology.orchestra.run.smithi135.stderr:WARNING: debug mode. Not for benchmarking or production 2023-08-24T10:04:40.413 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:40,413 seastar - Reactor backend: linux-aio 2023-08-24T10:04:40.431 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:40,431 [shard 1] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-08-24T10:04:40.431 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:40,431 [shard 1] seastar - IO queue uses 0.75ms latency goal for device 0 2023-08-24T10:04:40.432 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:40,431 [shard 1] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-08-24T10:04:40.439 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:40,438 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-08-24T10:04:40.441 INFO:teuthology.orchestra.run.smithi135.stderr:==80913==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-08-24T10:04:40.449 INFO:teuthology.orchestra.run.smithi135.stderr:==80913==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7fff4cc00000; bottom 0x7fb3b537e000; size: 0x004b97882000 (324664827904) 2023-08-24T10:04:40.450 INFO:teuthology.orchestra.run.smithi135.stderr:False positive error reports may follow 2023-08-24T10:04:40.450 INFO:teuthology.orchestra.run.smithi135.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-08-24T10:04:43.752 INFO:teuthology.orchestra.run.smithi135.stderr:----------------------------------------------------- 2023-08-24T10:04:43.753 INFO:teuthology.orchestra.run.smithi135.stderr:Suppressions used: 2023-08-24T10:04:43.753 INFO:teuthology.orchestra.run.smithi135.stderr: count bytes template 2023-08-24T10:04:43.753 INFO:teuthology.orchestra.run.smithi135.stderr: 1 8 MallocExtension::Register 2023-08-24T10:04:43.753 INFO:teuthology.orchestra.run.smithi135.stderr:----------------------------------------------------- 2023-08-24T10:04:43.753 INFO:teuthology.orchestra.run.smithi135.stderr: 2023-08-24T10:04:43.754 INFO:teuthology.orchestra.run.smithi135.stdout:created object store /var/lib/ceph/osd/ceph-2 for osd.2 fsid bf8addf4-f373-46ca-83a9-ccdf9773802d 2023-08-24T10:04:43.792 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-08-24T10:04:43.819 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-24T10:04:44.315 INFO:teuthology.orchestra.run.smithi135.stderr:WARNING: debug mode. Not for benchmarking or production 2023-08-24T10:04:44.315 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:44,315 seastar - Reactor backend: linux-aio 2023-08-24T10:04:44.332 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:44,331 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-08-24T10:04:44.332 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:44,331 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-08-24T10:04:44.332 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:44,331 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-08-24T10:04:44.332 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:44,331 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-08-24T10:04:44.334 INFO:teuthology.orchestra.run.smithi135.stderr:==81723==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-08-24T10:04:44.340 INFO:teuthology.orchestra.run.smithi135.stderr:==81723==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7fffcd62e000; bottom 0x7f8f48833000; size: 0x007084dfb000 (483265589248) 2023-08-24T10:04:44.340 INFO:teuthology.orchestra.run.smithi135.stderr:False positive error reports may follow 2023-08-24T10:04:44.340 INFO:teuthology.orchestra.run.smithi135.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-08-24T10:04:44.378 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:44,378 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2023-08-24T10:04:44.379 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:44,378 [shard 0] osd - get_early_config: set --smp 3 2023-08-24T10:04:44.859 INFO:teuthology.orchestra.run.smithi135.stderr:----------------------------------------------------- 2023-08-24T10:04:44.859 INFO:teuthology.orchestra.run.smithi135.stderr:Suppressions used: 2023-08-24T10:04:44.859 INFO:teuthology.orchestra.run.smithi135.stderr: count bytes template 2023-08-24T10:04:44.860 INFO:teuthology.orchestra.run.smithi135.stderr: 1 8 MallocExtension::Register 2023-08-24T10:04:44.860 INFO:teuthology.orchestra.run.smithi135.stderr:----------------------------------------------------- 2023-08-24T10:04:44.860 INFO:teuthology.orchestra.run.smithi135.stderr: 2023-08-24T10:04:44.919 INFO:teuthology.orchestra.run.smithi135.stderr:WARNING: debug mode. Not for benchmarking or production 2023-08-24T10:04:44.919 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:44,919 seastar - Reactor backend: linux-aio 2023-08-24T10:04:44.938 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:44,937 [shard 1] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-08-24T10:04:44.938 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:44,937 [shard 1] seastar - IO queue uses 0.75ms latency goal for device 0 2023-08-24T10:04:44.938 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:44,937 [shard 1] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-08-24T10:04:44.945 INFO:teuthology.orchestra.run.smithi135.stderr:INFO 2023-08-24 10:04:44,945 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-08-24T10:04:44.947 INFO:teuthology.orchestra.run.smithi135.stderr:==81721==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-08-24T10:04:44.953 INFO:teuthology.orchestra.run.smithi135.stderr:==81721==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7fffcd62e000; bottom 0x7f8f4661c000; size: 0x007087012000 (483301335040) 2023-08-24T10:04:44.953 INFO:teuthology.orchestra.run.smithi135.stderr:False positive error reports may follow 2023-08-24T10:04:44.954 INFO:teuthology.orchestra.run.smithi135.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-08-24T10:04:48.033 INFO:teuthology.orchestra.run.smithi135.stderr:----------------------------------------------------- 2023-08-24T10:04:48.033 INFO:teuthology.orchestra.run.smithi135.stderr:Suppressions used: 2023-08-24T10:04:48.033 INFO:teuthology.orchestra.run.smithi135.stderr: count bytes template 2023-08-24T10:04:48.034 INFO:teuthology.orchestra.run.smithi135.stderr: 1 8 MallocExtension::Register 2023-08-24T10:04:48.034 INFO:teuthology.orchestra.run.smithi135.stderr:----------------------------------------------------- 2023-08-24T10:04:48.034 INFO:teuthology.orchestra.run.smithi135.stderr: 2023-08-24T10:04:48.034 INFO:teuthology.orchestra.run.smithi135.stdout:created object store /var/lib/ceph/osd/ceph-3 for osd.3 fsid bf8addf4-f373-46ca-83a9-ccdf9773802d 2023-08-24T10:04:48.083 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-08-24T10:04:48.111 INFO:tasks.ceph:Reading keys from all nodes... 2023-08-24T10:04:48.112 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-08-24T10:04:48.112 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-08-24T10:04:48.148 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-08-24T10:04:48.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-08-24T10:04:48.221 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-24T10:04:48.221 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-08-24T10:04:48.248 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-24T10:04:48.248 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-08-24T10:04:48.314 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-24T10:04:48.314 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-08-24T10:04:48.381 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-08-24T10:04:48.381 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-08-24T10:04:48.398 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-24T10:04:48.399 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-08-24T10:04:48.435 INFO:tasks.ceph:Adding keys to all mons... 2023-08-24T10:04:48.436 DEBUG:teuthology.orchestra.run.smithi060:> sudo tee -a /etc/ceph/ceph.keyring 2023-08-24T10:04:48.472 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.0] 2023-08-24T10:04:48.473 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAtK+dkXEHvExAAt/u7NFGEOXw/DVATasm7Fw== 2023-08-24T10:04:48.473 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.1] 2023-08-24T10:04:48.473 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAyK+dke/LiCxAAF4/O9/8/pfa1KJh4zeXK0A== 2023-08-24T10:04:48.473 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr.x] 2023-08-24T10:04:48.473 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQArK+dkmG1eFBAAdxTAT+ngZm3UPK9CVljTnw== 2023-08-24T10:04:48.474 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.2] 2023-08-24T10:04:48.474 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQA4K+dkSDZ3HRAAvLlShpBDBF69rouniSJ4lw== 2023-08-24T10:04:48.474 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.3] 2023-08-24T10:04:48.474 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQA9K+dk08YNABAAhAfwJrZXgmCr8sDKvHec7g== 2023-08-24T10:04:48.474 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-08-24T10:04:48.474 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQArK+dkF0O7FxAA+KPLmXVz8Qxx7hDXXfEpuA== 2023-08-24T10:04:48.475 INFO:teuthology.orchestra.run.smithi060.stdout:[client.1] 2023-08-24T10:04:48.475 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQArK+dkfNjDGxAAifu3VsSMxU2Yvr/sdq9iYA== 2023-08-24T10:04:48.476 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-24T10:04:48.584 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-24T10:04:48.648 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-24T10:04:48.711 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-24T10:04:48.774 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-24T10:04:48.837 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-24T10:04:48.905 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-24T10:04:48.971 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-08-24T10:04:48.971 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-08-24T10:04:49.002 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-08-24T10:04:49.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.100+0000 7f0f51005980 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:49.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.100+0000 7f0f51005980 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:49.117 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-08-24T10:04:49.147 DEBUG:teuthology.orchestra.run.smithi060:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-08-24T10:04:49.202 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-08-24T10:04:49.202 INFO:tasks.ceph.mon.a:Restarting daemon 2023-08-24T10:04:49.203 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-08-24T10:04:49.244 INFO:tasks.ceph.mon.a:Started 2023-08-24T10:04:49.244 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-08-24T10:04:49.245 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-08-24T10:04:49.245 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-08-24T10:04:49.247 INFO:tasks.ceph.mgr.x:Started 2023-08-24T10:04:49.248 INFO:tasks.ceph:Setting crush tunables to default 2023-08-24T10:04:49.248 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph osd crush tunables default 2023-08-24T10:04:49.328 INFO:tasks.ceph.mon.a.smithi060.stderr:2023-08-24T10:04:49.327+0000 7fa572a05980 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:49.328 INFO:tasks.ceph.mon.a.smithi060.stderr:2023-08-24T10:04:49.327+0000 7fa572a05980 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:49.329 INFO:tasks.ceph.mon.a.smithi060.stderr:2023-08-24T10:04:49.327+0000 7fa572a05980 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:49.345 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:49.343+0000 7fe109564f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:49.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.354+0000 7ff19046c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:49.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.360+0000 7ff19046c640 1 Processor -- start 2023-08-24T10:04:49.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.360+0000 7ff19046c640 1 -- start start 2023-08-24T10:04:49.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.360+0000 7ff19046c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff188070410 0x7ff188098a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:49.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.360+0000 7ff19046c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff18806f720 con 0x7ff1881303c0 2023-08-24T10:04:49.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.360+0000 7ff19046c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff18806fb30 con 0x7ff188070410 2023-08-24T10:04:49.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.361+0000 7ff18e208640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff188070410 0x7ff188098a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.361+0000 7ff18e208640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff188070410 0x7ff188098a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53620/0 (socket says 172.21.15.60:53620) 2023-08-24T10:04:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.361+0000 7ff18e208640 1 -- 172.21.15.60:0/1975167155 learned_addr learned my addr 172.21.15.60:0/1975167155 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:04:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.361+0000 7ff18da07640 1 --1- 172.21.15.60:0/1975167155 >> v1:172.21.15.60:6789/0 conn(0x7ff1881303c0 0x7ff1881307d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49010/0 (socket says 172.21.15.60:49010) 2023-08-24T10:04:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.361+0000 7ff18e208640 1 -- 172.21.15.60:0/1975167155 >> v2:172.21.15.60:3300/0 conn(0x7ff188070410 msgr2=0x7ff188098a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-08-24T10:04:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.361+0000 7ff18e208640 1 -- 172.21.15.60:0/1975167155 >> v2:172.21.15.60:3300/0 conn(0x7ff188070410 msgr2=0x7ff188098a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-24T10:04:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.361+0000 7ff18e208640 1 --2- 172.21.15.60:0/1975167155 >> v2:172.21.15.60:3300/0 conn(0x7ff188070410 0x7ff188098a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-24T10:04:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.361+0000 7ff18e208640 1 --2- 172.21.15.60:0/1975167155 >> v2:172.21.15.60:3300/0 conn(0x7ff188070410 0x7ff188098a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:04:49.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.381+0000 7ff18d206640 1 -- 172.21.15.60:0/1975167155 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237561867 0 0) 0x7ff18806f720 con 0x7ff1881303c0 2023-08-24T10:04:49.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.381+0000 7ff18d206640 1 -- 172.21.15.60:0/1975167155 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1740034f0 con 0x7ff1881303c0 2023-08-24T10:04:49.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.381+0000 7ff18d206640 1 -- 172.21.15.60:0/1975167155 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1759882463 0 0) 0x7ff1740034f0 con 0x7ff1881303c0 2023-08-24T10:04:49.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.381+0000 7ff18d206640 1 -- 172.21.15.60:0/1975167155 >> v2:172.21.15.60:3300/0 conn(0x7ff188070410 msgr2=0x7ff188098a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:04:49.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.381+0000 7ff18d206640 1 --2- 172.21.15.60:0/1975167155 >> v2:172.21.15.60:3300/0 conn(0x7ff188070410 0x7ff188098a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:49.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.381+0000 7ff18d206640 1 -- 172.21.15.60:0/1975167155 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff188070820 con 0x7ff1881303c0 2023-08-24T10:04:49.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.381+0000 7ff18d206640 1 -- 172.21.15.60:0/1975167155 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (unknown 3178204431 0 0) 0x7ff1780015a0 con 0x7ff1881303c0 2023-08-24T10:04:49.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.381+0000 7ff18d206640 1 -- 172.21.15.60:0/1975167155 >> v1:172.21.15.60:6789/0 conn(0x7ff1881303c0 legacy=0x7ff1881307d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:49.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.381+0000 7ff18d206640 1 --2- 172.21.15.60:0/1975167155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff174003940 0x7ff174003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:49.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.381+0000 7ff18d206640 1 -- 172.21.15.60:0/1975167155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff174004460 con 0x7ff174003940 2023-08-24T10:04:49.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.382+0000 7ff18e208640 1 --2- 172.21.15.60:0/1975167155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff174003940 0x7ff174003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:49.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.382+0000 7ff18e208640 1 -- 172.21.15.60:0/1975167155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1740034f0 con 0x7ff174003940 2023-08-24T10:04:49.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.383+0000 7ff18e208640 1 --2- 172.21.15.60:0/1975167155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff174003940 0x7ff174003d90 crc :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d539aa3924c9caae server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:49.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.383+0000 7ff18d206640 1 -- 172.21.15.60:0/1975167155 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff18400c600 con 0x7ff174003940 2023-08-24T10:04:49.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.384+0000 7ff18d206640 1 -- 172.21.15.60:0/1975167155 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ff18400cb10 con 0x7ff174003940 2023-08-24T10:04:49.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.384+0000 7ff18d206640 1 -- 172.21.15.60:0/1975167155 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff18400cdb0 con 0x7ff174003940 2023-08-24T10:04:49.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.384+0000 7ff19046c640 1 -- 172.21.15.60:0/1975167155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff174003940 msgr2=0x7ff174003d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:49.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.384+0000 7ff19046c640 1 --2- 172.21.15.60:0/1975167155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff174003940 0x7ff174003d90 crc :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:49.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.384+0000 7ff19046c640 1 -- 172.21.15.60:0/1975167155 shutdown_connections 2023-08-24T10:04:49.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.384+0000 7ff19046c640 1 --2- 172.21.15.60:0/1975167155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff174003940 0x7ff174003d90 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:49.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.384+0000 7ff19046c640 1 --2- 172.21.15.60:0/1975167155 >> v2:172.21.15.60:3300/0 conn(0x7ff188070410 0x7ff188098a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:49.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.384+0000 7ff19046c640 1 -- 172.21.15.60:0/1975167155 >> 172.21.15.60:0/1975167155 conn(0x7ff18809efd0 msgr2=0x7ff18809f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:04:49.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.385+0000 7ff19046c640 1 -- 172.21.15.60:0/1975167155 shutdown_connections 2023-08-24T10:04:49.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.385+0000 7ff19046c640 1 -- 172.21.15.60:0/1975167155 wait complete. 2023-08-24T10:04:49.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.385+0000 7ff19046c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:49.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.386+0000 7ff19046c640 1 Processor -- start 2023-08-24T10:04:49.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.386+0000 7ff19046c640 1 -- start start 2023-08-24T10:04:49.389 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:49.385+0000 7fe109564f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:49.389 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:49.385+0000 7fe109564f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:49.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.386+0000 7ff19046c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff174003940 0x7ff188119960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:49.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.386+0000 7ff19046c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff188119640 con 0x7ff174003940 2023-08-24T10:04:49.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.387+0000 7ff18e208640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff174003940 0x7ff188119960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:49.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.387+0000 7ff18e208640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff174003940 0x7ff188119960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53630/0 (socket says 172.21.15.60:53630) 2023-08-24T10:04:49.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.387+0000 7ff18e208640 1 -- 172.21.15.60:0/1862369083 learned_addr learned my addr 172.21.15.60:0/1862369083 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:04:49.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.387+0000 7ff18e208640 1 -- 172.21.15.60:0/1862369083 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18811b5a0 con 0x7ff174003940 2023-08-24T10:04:49.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.387+0000 7ff18e208640 1 --2- 172.21.15.60:0/1862369083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff174003940 0x7ff188119960 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:49.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.387+0000 7ff17effd640 1 -- 172.21.15.60:0/1862369083 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff18400cc80 con 0x7ff174003940 2023-08-24T10:04:49.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.387+0000 7ff17effd640 1 -- 172.21.15.60:0/1862369083 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ff18400cde0 con 0x7ff174003940 2023-08-24T10:04:49.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.387+0000 7ff19046c640 1 -- 172.21.15.60:0/1862369083 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1881192a0 con 0x7ff174003940 2023-08-24T10:04:49.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.387+0000 7ff17effd640 1 -- 172.21.15.60:0/1862369083 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff184018580 con 0x7ff174003940 2023-08-24T10:04:49.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.387+0000 7ff19046c640 1 -- 172.21.15.60:0/1862369083 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff188118f20 con 0x7ff174003940 2023-08-24T10:04:49.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.387+0000 7ff17effd640 1 -- 172.21.15.60:0/1862369083 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (crc 0 0 0) 0x7ff184018730 con 0x7ff174003940 2023-08-24T10:04:49.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.387+0000 7ff17effd640 1 -- 172.21.15.60:0/1862369083 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (crc 0 0 0) 0x7ff184015b20 con 0x7ff174003940 2023-08-24T10:04:49.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.388+0000 7ff19046c640 1 -- 172.21.15.60:0/1862369083 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff188061800 con 0x7ff174003940 2023-08-24T10:04:49.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.390+0000 7ff17effd640 1 -- 172.21.15.60:0/1862369083 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (crc 0 0 0) 0x7ff18400f040 con 0x7ff174003940 2023-08-24T10:04:49.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.457+0000 7ff19046c640 1 -- 172.21.15.60:0/1862369083 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff18811b7f0 con 0x7ff174003940 2023-08-24T10:04:49.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.459+0000 7ff17effd640 1 -- 172.21.15.60:0/1862369083 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (crc 0 0 0) 0x7ff184015d80 con 0x7ff174003940 2023-08-24T10:04:49.461 INFO:teuthology.orchestra.run.smithi060.stderr:adjusted tunables profile to default 2023-08-24T10:04:49.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.460+0000 7ff19046c640 1 -- 172.21.15.60:0/1862369083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff174003940 msgr2=0x7ff188119960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:49.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.460+0000 7ff19046c640 1 --2- 172.21.15.60:0/1862369083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff174003940 0x7ff188119960 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:49.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.460+0000 7ff19046c640 1 -- 172.21.15.60:0/1862369083 shutdown_connections 2023-08-24T10:04:49.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.460+0000 7ff19046c640 1 --2- 172.21.15.60:0/1862369083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff174003940 0x7ff188119960 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:49.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.460+0000 7ff19046c640 1 -- 172.21.15.60:0/1862369083 >> 172.21.15.60:0/1862369083 conn(0x7ff18809efd0 msgr2=0x7ff18812b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:04:49.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.460+0000 7ff19046c640 1 -- 172.21.15.60:0/1862369083 shutdown_connections 2023-08-24T10:04:49.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.460+0000 7ff19046c640 1 -- 172.21.15.60:0/1862369083 wait complete. 2023-08-24T10:04:49.472 INFO:tasks.ceph:check_enable_crimson: True 2023-08-24T10:04:49.473 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2023-08-24T10:04:49.474 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2023-08-24T10:04:49.529 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:49.528+0000 7fe109564f00 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-24T10:04:49.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.578+0000 7f77dbaec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:49.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.583+0000 7f77dbaec640 1 Processor -- start 2023-08-24T10:04:49.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.583+0000 7f77dbaec640 1 -- start start 2023-08-24T10:04:49.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.584+0000 7f77dbaec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f77d412fd50 0x7f77d4134580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:49.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.584+0000 7f77dbaec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d4099eb0 con 0x7f77d409a350 2023-08-24T10:04:49.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.584+0000 7f77dbaec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d4130160 con 0x7f77d412fd50 2023-08-24T10:04:49.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.584+0000 7f77d9087640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f77d412fd50 0x7f77d4134580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:49.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.584+0000 7f77d9888640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f77d409a350 0x7f77d412f640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49024/0 (socket says 172.21.15.60:49024) 2023-08-24T10:04:49.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.584+0000 7f77d9888640 1 -- 172.21.15.60:0/4108934731 learned_addr learned my addr 172.21.15.60:0/4108934731 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:04:49.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.585+0000 7f77d9087640 1 -- 172.21.15.60:0/4108934731 >> v1:172.21.15.60:6789/0 conn(0x7f77d409a350 legacy=0x7f77d412f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:49.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.585+0000 7f77d9087640 1 -- 172.21.15.60:0/4108934731 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d409a810 con 0x7f77d412fd50 2023-08-24T10:04:49.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.585+0000 7f77d8886640 1 -- 172.21.15.60:0/4108934731 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887618860 0 0) 0x7f77d4099eb0 con 0x7f77d409a350 2023-08-24T10:04:49.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.585+0000 7f77d9087640 1 --2- 172.21.15.60:0/4108934731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d412fd50 0x7f77d4134580 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312637cd439aff9e server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:49.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.585+0000 7f77d8886640 1 -- 172.21.15.60:0/4108934731 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f77c8009ce0 con 0x7f77d412fd50 2023-08-24T10:04:49.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.585+0000 7f77d8886640 1 -- 172.21.15.60:0/4108934731 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f77c8009e40 con 0x7f77d412fd50 2023-08-24T10:04:49.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.585+0000 7f77d8886640 1 -- 172.21.15.60:0/4108934731 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f77c800e360 con 0x7f77d412fd50 2023-08-24T10:04:49.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.586+0000 7f77dbaec640 1 -- 172.21.15.60:0/4108934731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d412fd50 msgr2=0x7f77d4134580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:49.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.586+0000 7f77dbaec640 1 --2- 172.21.15.60:0/4108934731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d412fd50 0x7f77d4134580 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:49.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.586+0000 7f77dbaec640 1 -- 172.21.15.60:0/4108934731 shutdown_connections 2023-08-24T10:04:49.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.586+0000 7f77dbaec640 1 --2- 172.21.15.60:0/4108934731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d412fd50 0x7f77d4134580 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:49.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.586+0000 7f77dbaec640 1 -- 172.21.15.60:0/4108934731 >> 172.21.15.60:0/4108934731 conn(0x7f77d409d930 msgr2=0x7f77d409dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:04:49.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.586+0000 7f77dbaec640 1 -- 172.21.15.60:0/4108934731 shutdown_connections 2023-08-24T10:04:49.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.586+0000 7f77dbaec640 1 -- 172.21.15.60:0/4108934731 wait complete. 2023-08-24T10:04:49.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.586+0000 7f77dbaec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:49.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.587+0000 7f77dbaec640 1 Processor -- start 2023-08-24T10:04:49.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.587+0000 7f77dbaec640 1 -- start start 2023-08-24T10:04:49.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.587+0000 7f77dbaec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d412fd50 0x7f77d41c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:49.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.587+0000 7f77dbaec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d4122880 con 0x7f77d412fd50 2023-08-24T10:04:49.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.588+0000 7f77d9888640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d412fd50 0x7f77d41c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:49.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.588+0000 7f77d9888640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d412fd50 0x7f77d41c46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53648/0 (socket says 172.21.15.60:53648) 2023-08-24T10:04:49.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.588+0000 7f77d9888640 1 -- 172.21.15.60:0/2637868663 learned_addr learned my addr 172.21.15.60:0/2637868663 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:04:49.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.588+0000 7f77d9888640 1 -- 172.21.15.60:0/2637868663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d4122ab0 con 0x7f77d412fd50 2023-08-24T10:04:49.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.588+0000 7f77d9888640 1 --2- 172.21.15.60:0/2637868663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d412fd50 0x7f77d41c46b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:49.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.588+0000 7f77d27fc640 1 -- 172.21.15.60:0/2637868663 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f77cc004af0 con 0x7f77d412fd50 2023-08-24T10:04:49.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.588+0000 7f77d27fc640 1 -- 172.21.15.60:0/2637868663 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f77cc004c50 con 0x7f77d412fd50 2023-08-24T10:04:49.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.588+0000 7f77dbaec640 1 -- 172.21.15.60:0/2637868663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d412f030 con 0x7f77d412fd50 2023-08-24T10:04:49.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.588+0000 7f77d27fc640 1 -- 172.21.15.60:0/2637868663 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f77cc007120 con 0x7f77d412fd50 2023-08-24T10:04:49.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.588+0000 7f77dbaec640 1 -- 172.21.15.60:0/2637868663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d4122570 con 0x7f77d412fd50 2023-08-24T10:04:49.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.588+0000 7f77d27fc640 1 -- 172.21.15.60:0/2637868663 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (crc 0 0 0) 0x7f77cc00c4e0 con 0x7f77d412fd50 2023-08-24T10:04:49.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.588+0000 7f77d27fc640 1 -- 172.21.15.60:0/2637868663 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (crc 0 0 0) 0x7f77cc00f610 con 0x7f77d412fd50 2023-08-24T10:04:49.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.589+0000 7f77dbaec640 1 -- 172.21.15.60:0/2637868663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d4120160 con 0x7f77d412fd50 2023-08-24T10:04:49.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.591+0000 7f77d27fc640 1 -- 172.21.15.60:0/2637868663 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (crc 0 0 0) 0x7f77cc00c970 con 0x7f77d412fd50 2023-08-24T10:04:49.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:49.667+0000 7f77dbaec640 1 -- 172.21.15.60:0/2637868663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f77d412eea0 con 0x7f77d412fd50 2023-08-24T10:04:49.668 INFO:tasks.ceph.mon.a.smithi060.stderr:2023-08-24T10:04:49.667+0000 7fa569df1640 -1 WARNING: experimental feature 'crimson' is enabled 2023-08-24T10:04:49.668 INFO:tasks.ceph.mon.a.smithi060.stderr:Please be aware that this feature is experimental, untested, 2023-08-24T10:04:49.669 INFO:tasks.ceph.mon.a.smithi060.stderr:unsupported, and may result in data corruption, data loss, 2023-08-24T10:04:49.669 INFO:tasks.ceph.mon.a.smithi060.stderr:and/or irreparable damage to your cluster. Do not use 2023-08-24T10:04:49.669 INFO:tasks.ceph.mon.a.smithi060.stderr:feature with important data. 2023-08-24T10:04:49.669 INFO:tasks.ceph.mon.a.smithi060.stderr: 2023-08-24T10:04:50.325 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:50.324+0000 7fe109564f00 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-24T10:04:50.460 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:50.459+0000 7fe109564f00 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-24T10:04:50.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:50.460+0000 7f77d27fc640 1 -- 172.21.15.60:0/2637868663 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true}]=0 v3) v1 ==== 99+0+0 (crc 0 0 0) 0x7f77cc00cb10 con 0x7f77d412fd50 2023-08-24T10:04:50.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:50.461+0000 7f77dbaec640 1 -- 172.21.15.60:0/2637868663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d412fd50 msgr2=0x7f77d41c46b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:50.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:50.461+0000 7f77dbaec640 1 --2- 172.21.15.60:0/2637868663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d412fd50 0x7f77d41c46b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:50.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:50.461+0000 7f77dbaec640 1 -- 172.21.15.60:0/2637868663 shutdown_connections 2023-08-24T10:04:50.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:50.461+0000 7f77dbaec640 1 --2- 172.21.15.60:0/2637868663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77d412fd50 0x7f77d41c46b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:50.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:50.461+0000 7f77dbaec640 1 -- 172.21.15.60:0/2637868663 >> 172.21.15.60:0/2637868663 conn(0x7f77d409d930 msgr2=0x7f77d409ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:04:50.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:50.461+0000 7f77dbaec640 1 -- 172.21.15.60:0/2637868663 shutdown_connections 2023-08-24T10:04:50.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:50.461+0000 7f77dbaec640 1 -- 172.21.15.60:0/2637868663 wait complete. 2023-08-24T10:04:50.474 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-08-24T10:04:50.475 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-08-24T10:04:50.475 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-08-24T10:04:50.508 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:50.506+0000 7fe109564f00 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-24T10:04:50.509 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-08-24T10:04:50.509 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-08-24T10:04:50.551 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:50.549+0000 7fe109564f00 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-24T10:04:50.585 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-24T10:04:50.585 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-08-24T10:04:50.614 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2023-08-24T10:04:50.614 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-08-24T10:04:50.681 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new 72632e8e-7a21-476f-91d4-c34b43117b99 0 2023-08-24T10:04:50.828 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.821+0000 7f062ad5e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:50.829 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.827+0000 7f062ad5e640 1 Processor -- start 2023-08-24T10:04:50.829 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.827+0000 7f062ad5e640 1 -- start start 2023-08-24T10:04:50.829 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.827+0000 7f062ad5e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06241117e0 0x7f0624111bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:50.829 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.828+0000 7f062ad5e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0624116cb0 con 0x7f0624116190 2023-08-24T10:04:50.829 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.828+0000 7f062ad5e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0624116e40 con 0x7f06241117e0 2023-08-24T10:04:50.830 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.828+0000 7f0628afa640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06241117e0 0x7f0624111bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:50.830 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.828+0000 7f0623fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0624116190 0x7f06241165a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.135:43992/0 (socket says 172.21.15.135:43992) 2023-08-24T10:04:50.830 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.828+0000 7f0628afa640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06241117e0 0x7f0624111bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.135:43926/0 (socket says 172.21.15.135:43926) 2023-08-24T10:04:50.831 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.828+0000 7f0623fff640 1 -- 172.21.15.135:0/3003286595 learned_addr learned my addr 172.21.15.135:0/3003286595 (peer_addr_for_me v1:172.21.15.135:0/0) 2023-08-24T10:04:50.831 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.828+0000 7f0628afa640 1 -- 172.21.15.135:0/3003286595 >> v1:172.21.15.60:6789/0 conn(0x7f0624116190 legacy=0x7f06241165a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:50.831 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.828+0000 7f0628afa640 1 -- 172.21.15.135:0/3003286595 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0624116fb0 con 0x7f06241117e0 2023-08-24T10:04:50.831 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.829+0000 7f0628afa640 1 --2- 172.21.15.135:0/3003286595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06241117e0 0x7f0624111bf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d9cc9b06ab2a0a server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:50.831 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.829+0000 7f06237fe640 1 -- 172.21.15.135:0/3003286595 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0618009ce0 con 0x7f06241117e0 2023-08-24T10:04:50.831 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.829+0000 7f06237fe640 1 -- 172.21.15.135:0/3003286595 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0618009e40 con 0x7f06241117e0 2023-08-24T10:04:50.831 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.829+0000 7f06237fe640 1 -- 172.21.15.135:0/3003286595 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f061800e360 con 0x7f06241117e0 2023-08-24T10:04:50.832 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.829+0000 7f062ad5e640 1 -- 172.21.15.135:0/3003286595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06241117e0 msgr2=0x7f0624111bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:50.832 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.829+0000 7f062ad5e640 1 --2- 172.21.15.135:0/3003286595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06241117e0 0x7f0624111bf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:50.832 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.829+0000 7f062ad5e640 1 -- 172.21.15.135:0/3003286595 shutdown_connections 2023-08-24T10:04:50.832 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.829+0000 7f062ad5e640 1 --2- 172.21.15.135:0/3003286595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06241117e0 0x7f0624111bf0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:50.837 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.829+0000 7f062ad5e640 1 -- 172.21.15.135:0/3003286595 >> 172.21.15.135:0/3003286595 conn(0x7f062410d290 msgr2=0x7f062410f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:04:50.837 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.830+0000 7f062ad5e640 1 -- 172.21.15.135:0/3003286595 shutdown_connections 2023-08-24T10:04:50.837 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.830+0000 7f062ad5e640 1 -- 172.21.15.135:0/3003286595 wait complete. 2023-08-24T10:04:50.838 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.830+0000 7f062ad5e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:50.838 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.830+0000 7f062ad5e640 1 Processor -- start 2023-08-24T10:04:50.838 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.830+0000 7f062ad5e640 1 -- start start 2023-08-24T10:04:50.838 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.831+0000 7f062ad5e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06241117e0 0x7f06241a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:50.838 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.831+0000 7f062ad5e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06241a6230 con 0x7f06241117e0 2023-08-24T10:04:50.838 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.831+0000 7f0628afa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06241117e0 0x7f06241a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:50.838 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.831+0000 7f0628afa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06241117e0 0x7f06241a5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.135:43930/0 (socket says 172.21.15.135:43930) 2023-08-24T10:04:50.838 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.831+0000 7f0628afa640 1 -- 172.21.15.135:0/4115456963 learned_addr learned my addr 172.21.15.135:0/4115456963 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-24T10:04:50.839 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.831+0000 7f0628afa640 1 -- 172.21.15.135:0/4115456963 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06241a6370 con 0x7f06241117e0 2023-08-24T10:04:50.839 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.832+0000 7f0628afa640 1 --2- 172.21.15.135:0/4115456963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06241117e0 0x7f06241a5c60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:50.839 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.832+0000 7f06217fa640 1 -- 172.21.15.135:0/4115456963 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0618014540 con 0x7f06241117e0 2023-08-24T10:04:50.839 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.832+0000 7f06217fa640 1 -- 172.21.15.135:0/4115456963 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f06180146a0 con 0x7f06241117e0 2023-08-24T10:04:50.839 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.832+0000 7f06217fa640 1 -- 172.21.15.135:0/4115456963 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0618014940 con 0x7f06241117e0 2023-08-24T10:04:50.839 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.832+0000 7f062ad5e640 1 -- 172.21.15.135:0/4115456963 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06241a65c0 con 0x7f06241117e0 2023-08-24T10:04:50.839 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.832+0000 7f062ad5e640 1 -- 172.21.15.135:0/4115456963 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06241aa4c0 con 0x7f06241117e0 2023-08-24T10:04:50.840 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.832+0000 7f06217fa640 1 -- 172.21.15.135:0/4115456963 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (crc 0 0 0) 0x7f0618013400 con 0x7f06241117e0 2023-08-24T10:04:50.840 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.832+0000 7f06217fa640 1 -- 172.21.15.135:0/4115456963 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 747+0+0 (crc 0 0 0) 0x7f0618019040 con 0x7f06241117e0 2023-08-24T10:04:50.840 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.833+0000 7f062ad5e640 1 -- 172.21.15.135:0/4115456963 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06241a67b0 con 0x7f06241117e0 2023-08-24T10:04:50.840 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.835+0000 7f06217fa640 1 -- 172.21.15.135:0/4115456963 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (crc 0 0 0) 0x7f0618013940 con 0x7f06241117e0 2023-08-24T10:04:50.908 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.907+0000 7f062ad5e640 1 -- 172.21.15.135:0/4115456963 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd new", "uuid": "72632e8e-7a21-476f-91d4-c34b43117b99", "id": 0} v 0) v1 -- 0x7f06241ab0e0 con 0x7f06241117e0 2023-08-24T10:04:50.911 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.910+0000 7f06217fa640 1 -- 172.21.15.135:0/4115456963 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "72632e8e-7a21-476f-91d4-c34b43117b99", "id": 0}]=0 v4) v1 ==== 112+0+2 (crc 0 0 0) 0x7f0618013c30 con 0x7f06241117e0 2023-08-24T10:04:50.911 INFO:teuthology.orchestra.run.smithi135.stdout:0 2023-08-24T10:04:50.912 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.911+0000 7f062ad5e640 1 -- 172.21.15.135:0/4115456963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06241117e0 msgr2=0x7f06241a5c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:50.912 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.911+0000 7f062ad5e640 1 --2- 172.21.15.135:0/4115456963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06241117e0 0x7f06241a5c60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:50.912 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.911+0000 7f062ad5e640 1 -- 172.21.15.135:0/4115456963 shutdown_connections 2023-08-24T10:04:50.912 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.911+0000 7f062ad5e640 1 --2- 172.21.15.135:0/4115456963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06241117e0 0x7f06241a5c60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:50.913 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.911+0000 7f062ad5e640 1 -- 172.21.15.135:0/4115456963 >> 172.21.15.135:0/4115456963 conn(0x7f062410d290 msgr2=0x7f06241171a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:04:50.913 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.911+0000 7f062ad5e640 1 -- 172.21.15.135:0/4115456963 shutdown_connections 2023-08-24T10:04:50.914 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:50.911+0000 7f062ad5e640 1 -- 172.21.15.135:0/4115456963 wait complete. 2023-08-24T10:04:50.922 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new 3157bd05-35e6-4d2c-9e8b-ec16937c5e5c 1 2023-08-24T10:04:51.030 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.022+0000 7f4795e63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:51.030 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.028+0000 7f4795e63640 1 Processor -- start 2023-08-24T10:04:51.030 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.028+0000 7f4795e63640 1 -- start start 2023-08-24T10:04:51.030 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.029+0000 7f4795e63640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f479011a300 0x7f479011c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:51.030 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.029+0000 7f4795e63640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f479011cd50 con 0x7f47901117e0 2023-08-24T10:04:51.031 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.029+0000 7f4795e63640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479011df10 con 0x7f479011a300 2023-08-24T10:04:51.032 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.029+0000 7f478effd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f479011a300 0x7f479011c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:51.033 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.029+0000 7f478effd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f479011a300 0x7f479011c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.135:43942/0 (socket says 172.21.15.135:43942) 2023-08-24T10:04:51.034 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.029+0000 7f478f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f47901117e0 0x7f4790111bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.135:43996/0 (socket says 172.21.15.135:43996) 2023-08-24T10:04:51.034 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.029+0000 7f478effd640 1 -- 172.21.15.135:0/2680685839 learned_addr learned my addr 172.21.15.135:0/2680685839 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-24T10:04:51.034 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.029+0000 7f478effd640 1 -- 172.21.15.135:0/2680685839 >> v1:172.21.15.60:6789/0 conn(0x7f47901117e0 legacy=0x7f4790111bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:51.034 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.029+0000 7f478effd640 1 -- 172.21.15.135:0/2680685839 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479011e080 con 0x7f479011a300 2023-08-24T10:04:51.034 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.029+0000 7f478effd640 1 --2- 172.21.15.135:0/2680685839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f479011a300 0x7f479011c720 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a072b50713e7c2f server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:51.034 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.030+0000 7f478e7fc640 1 -- 172.21.15.135:0/2680685839 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4784009dd0 con 0x7f479011a300 2023-08-24T10:04:51.035 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.030+0000 7f478e7fc640 1 -- 172.21.15.135:0/2680685839 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f478400e230 con 0x7f479011a300 2023-08-24T10:04:51.035 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.030+0000 7f478e7fc640 1 -- 172.21.15.135:0/2680685839 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f478400e4d0 con 0x7f479011a300 2023-08-24T10:04:51.035 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.030+0000 7f4795e63640 1 -- 172.21.15.135:0/2680685839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f479011a300 msgr2=0x7f479011c720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:51.035 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.030+0000 7f4795e63640 1 --2- 172.21.15.135:0/2680685839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f479011a300 0x7f479011c720 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:51.035 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.030+0000 7f4795e63640 1 -- 172.21.15.135:0/2680685839 shutdown_connections 2023-08-24T10:04:51.035 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.030+0000 7f4795e63640 1 --2- 172.21.15.135:0/2680685839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f479011a300 0x7f479011c720 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:51.035 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.030+0000 7f4795e63640 1 -- 172.21.15.135:0/2680685839 >> 172.21.15.135:0/2680685839 conn(0x7f479010d290 msgr2=0x7f479010f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:04:51.035 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.030+0000 7f4795e63640 1 -- 172.21.15.135:0/2680685839 shutdown_connections 2023-08-24T10:04:51.036 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.030+0000 7f4795e63640 1 -- 172.21.15.135:0/2680685839 wait complete. 2023-08-24T10:04:51.036 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.031+0000 7f4795e63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:51.036 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.031+0000 7f4795e63640 1 Processor -- start 2023-08-24T10:04:51.037 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.031+0000 7f4795e63640 1 -- start start 2023-08-24T10:04:51.038 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.032+0000 7f4795e63640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47901117e0 0x7f47901a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:51.038 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.032+0000 7f4795e63640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47901a62e0 con 0x7f47901117e0 2023-08-24T10:04:51.038 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.032+0000 7f478f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47901117e0 0x7f47901a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:51.038 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.032+0000 7f478f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47901117e0 0x7f47901a5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.135:43954/0 (socket says 172.21.15.135:43954) 2023-08-24T10:04:51.038 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.032+0000 7f478f7fe640 1 -- 172.21.15.135:0/1700851530 learned_addr learned my addr 172.21.15.135:0/1700851530 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-24T10:04:51.038 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.032+0000 7f478f7fe640 1 -- 172.21.15.135:0/1700851530 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47901a6420 con 0x7f47901117e0 2023-08-24T10:04:51.039 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.032+0000 7f478f7fe640 1 --2- 172.21.15.135:0/1700851530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47901117e0 0x7f47901a5da0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:51.039 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.032+0000 7f4777fff640 1 -- 172.21.15.135:0/1700851530 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f47800071e0 con 0x7f47901117e0 2023-08-24T10:04:51.039 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.032+0000 7f4777fff640 1 -- 172.21.15.135:0/1700851530 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4780007340 con 0x7f47901117e0 2023-08-24T10:04:51.039 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.032+0000 7f4777fff640 1 -- 172.21.15.135:0/1700851530 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f47800075e0 con 0x7f47901117e0 2023-08-24T10:04:51.039 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.032+0000 7f4795e63640 1 -- 172.21.15.135:0/1700851530 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47901a66d0 con 0x7f47901117e0 2023-08-24T10:04:51.039 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.033+0000 7f4795e63640 1 -- 172.21.15.135:0/1700851530 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47901aa520 con 0x7f47901117e0 2023-08-24T10:04:51.039 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.033+0000 7f4777fff640 1 -- 172.21.15.135:0/1700851530 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (crc 0 0 0) 0x7f4780005ae0 con 0x7f47901117e0 2023-08-24T10:04:51.040 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.033+0000 7f4777fff640 1 -- 172.21.15.135:0/1700851530 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 862+0+0 (crc 0 0 0) 0x7f478000f690 con 0x7f47901117e0 2023-08-24T10:04:51.040 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.033+0000 7f4795e63640 1 -- 172.21.15.135:0/1700851530 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4790061800 con 0x7f47901117e0 2023-08-24T10:04:51.040 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.036+0000 7f4777fff640 1 -- 172.21.15.135:0/1700851530 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (crc 0 0 0) 0x7f478000c780 con 0x7f47901117e0 2023-08-24T10:04:51.116 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.109+0000 7f4795e63640 1 -- 172.21.15.135:0/1700851530 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3157bd05-35e6-4d2c-9e8b-ec16937c5e5c", "id": 1} v 0) v1 -- 0x7f479011a300 con 0x7f47901117e0 2023-08-24T10:04:51.117 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.112+0000 7f4777fff640 1 -- 172.21.15.135:0/1700851530 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3157bd05-35e6-4d2c-9e8b-ec16937c5e5c", "id": 1}]=0 v5) v1 ==== 112+0+2 (crc 0 0 0) 0x7f478000ca70 con 0x7f47901117e0 2023-08-24T10:04:51.117 INFO:teuthology.orchestra.run.smithi135.stdout:1 2023-08-24T10:04:51.118 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.113+0000 7f4795e63640 1 -- 172.21.15.135:0/1700851530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47901117e0 msgr2=0x7f47901a5da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:51.118 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.113+0000 7f4795e63640 1 --2- 172.21.15.135:0/1700851530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47901117e0 0x7f47901a5da0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:51.118 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.113+0000 7f4795e63640 1 -- 172.21.15.135:0/1700851530 shutdown_connections 2023-08-24T10:04:51.118 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.113+0000 7f4795e63640 1 --2- 172.21.15.135:0/1700851530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47901117e0 0x7f47901a5da0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:51.118 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.113+0000 7f4795e63640 1 -- 172.21.15.135:0/1700851530 >> 172.21.15.135:0/1700851530 conn(0x7f479010d290 msgr2=0x7f4790113690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:04:51.118 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.113+0000 7f4795e63640 1 -- 172.21.15.135:0/1700851530 shutdown_connections 2023-08-24T10:04:51.119 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.113+0000 7f4795e63640 1 -- 172.21.15.135:0/1700851530 wait complete. 2023-08-24T10:04:51.125 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new 77ec2ee0-440e-41ae-ab0b-0427531dfa8a 2 2023-08-24T10:04:51.230 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.223+0000 7fa7204ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:51.230 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.228+0000 7fa7204ac640 1 Processor -- start 2023-08-24T10:04:51.230 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.228+0000 7fa7204ac640 1 -- start start 2023-08-24T10:04:51.231 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.229+0000 7fa7204ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa7181116e0 0x7fa718111af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:51.231 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.229+0000 7fa7204ac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7181122b0 con 0x7fa718116180 2023-08-24T10:04:51.231 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.229+0000 7fa7204ac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa718116ca0 con 0x7fa7181116e0 2023-08-24T10:04:51.233 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.229+0000 7fa71e248640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa7181116e0 0x7fa718111af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:51.234 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.229+0000 7fa71e248640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa7181116e0 0x7fa718111af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.135:43958/0 (socket says 172.21.15.135:43958) 2023-08-24T10:04:51.234 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.229+0000 7fa71e248640 1 -- 172.21.15.135:0/995348670 learned_addr learned my addr 172.21.15.135:0/995348670 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-24T10:04:51.234 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.229+0000 7fa71da47640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa718116180 0x7fa718116590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.135:44008/0 (socket says 172.21.15.135:44008) 2023-08-24T10:04:51.235 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.230+0000 7fa71e248640 1 -- 172.21.15.135:0/995348670 >> v1:172.21.15.60:6789/0 conn(0x7fa718116180 legacy=0x7fa718116590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:51.235 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.230+0000 7fa71e248640 1 -- 172.21.15.135:0/995348670 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa718116e10 con 0x7fa7181116e0 2023-08-24T10:04:51.235 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.230+0000 7fa71e248640 1 --2- 172.21.15.135:0/995348670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7181116e0 0x7fa718111af0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514d76af24213594 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:51.236 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.230+0000 7fa71d246640 1 -- 172.21.15.135:0/995348670 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa708009dd0 con 0x7fa7181116e0 2023-08-24T10:04:51.236 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.230+0000 7fa71d246640 1 -- 172.21.15.135:0/995348670 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa70800e230 con 0x7fa7181116e0 2023-08-24T10:04:51.236 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.230+0000 7fa71d246640 1 -- 172.21.15.135:0/995348670 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa70800e4d0 con 0x7fa7181116e0 2023-08-24T10:04:51.237 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.231+0000 7fa7204ac640 1 -- 172.21.15.135:0/995348670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7181116e0 msgr2=0x7fa718111af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:51.237 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.231+0000 7fa7204ac640 1 --2- 172.21.15.135:0/995348670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7181116e0 0x7fa718111af0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:51.237 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.231+0000 7fa7204ac640 1 -- 172.21.15.135:0/995348670 shutdown_connections 2023-08-24T10:04:51.238 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.231+0000 7fa7204ac640 1 --2- 172.21.15.135:0/995348670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7181116e0 0x7fa718111af0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:51.239 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.231+0000 7fa7204ac640 1 -- 172.21.15.135:0/995348670 >> 172.21.15.135:0/995348670 conn(0x7fa71810d290 msgr2=0x7fa71810f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:04:51.239 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.231+0000 7fa7204ac640 1 -- 172.21.15.135:0/995348670 shutdown_connections 2023-08-24T10:04:51.239 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.231+0000 7fa7204ac640 1 -- 172.21.15.135:0/995348670 wait complete. 2023-08-24T10:04:51.240 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.231+0000 7fa7204ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:51.240 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.231+0000 7fa7204ac640 1 Processor -- start 2023-08-24T10:04:51.240 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.231+0000 7fa7204ac640 1 -- start start 2023-08-24T10:04:51.241 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.232+0000 7fa7204ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7181116e0 0x7fa7181a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:51.241 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.232+0000 7fa7204ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7181a62f0 con 0x7fa7181116e0 2023-08-24T10:04:51.241 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.232+0000 7fa71e248640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7181116e0 0x7fa7181a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:51.241 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.232+0000 7fa71e248640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7181116e0 0x7fa7181a5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.135:43962/0 (socket says 172.21.15.135:43962) 2023-08-24T10:04:51.242 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.232+0000 7fa71e248640 1 -- 172.21.15.135:0/2534222401 learned_addr learned my addr 172.21.15.135:0/2534222401 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-24T10:04:51.243 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.232+0000 7fa71e248640 1 -- 172.21.15.135:0/2534222401 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7181a6430 con 0x7fa7181116e0 2023-08-24T10:04:51.243 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.233+0000 7fa71e248640 1 --2- 172.21.15.135:0/2534222401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7181116e0 0x7fa7181a5d20 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:51.244 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.233+0000 7fa70effd640 1 -- 172.21.15.135:0/2534222401 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa708014540 con 0x7fa7181116e0 2023-08-24T10:04:51.244 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.233+0000 7fa70effd640 1 -- 172.21.15.135:0/2534222401 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa7080146a0 con 0x7fa7181116e0 2023-08-24T10:04:51.244 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.233+0000 7fa7204ac640 1 -- 172.21.15.135:0/2534222401 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7181a6680 con 0x7fa7181116e0 2023-08-24T10:04:51.244 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.233+0000 7fa70effd640 1 -- 172.21.15.135:0/2534222401 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa708014940 con 0x7fa7181116e0 2023-08-24T10:04:51.244 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.233+0000 7fa7204ac640 1 -- 172.21.15.135:0/2534222401 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7181aa5b0 con 0x7fa7181116e0 2023-08-24T10:04:51.245 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.233+0000 7fa70effd640 1 -- 172.21.15.135:0/2534222401 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (crc 0 0 0) 0x7fa708013400 con 0x7fa7181116e0 2023-08-24T10:04:51.245 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.233+0000 7fa70effd640 1 -- 172.21.15.135:0/2534222401 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 977+0+0 (crc 0 0 0) 0x7fa708019040 con 0x7fa7181116e0 2023-08-24T10:04:51.245 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.234+0000 7fa7204ac640 1 -- 172.21.15.135:0/2534222401 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa718061800 con 0x7fa7181116e0 2023-08-24T10:04:51.245 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.236+0000 7fa70effd640 1 -- 172.21.15.135:0/2534222401 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (crc 0 0 0) 0x7fa708013890 con 0x7fa7181116e0 2023-08-24T10:04:51.307 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.306+0000 7fa7204ac640 1 -- 172.21.15.135:0/2534222401 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd new", "uuid": "77ec2ee0-440e-41ae-ab0b-0427531dfa8a", "id": 2} v 0) v1 -- 0x7fa718116180 con 0x7fa7181116e0 2023-08-24T10:04:51.308 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:51.307+0000 7fe109564f00 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-24T10:04:51.309 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.308+0000 7fa70effd640 1 -- 172.21.15.135:0/2534222401 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "77ec2ee0-440e-41ae-ab0b-0427531dfa8a", "id": 2}]=0 v6) v1 ==== 112+0+2 (crc 0 0 0) 0x7fa708013ba0 con 0x7fa7181116e0 2023-08-24T10:04:51.310 INFO:teuthology.orchestra.run.smithi135.stdout:2 2023-08-24T10:04:51.311 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.309+0000 7fa7204ac640 1 -- 172.21.15.135:0/2534222401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7181116e0 msgr2=0x7fa7181a5d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:51.311 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.309+0000 7fa7204ac640 1 --2- 172.21.15.135:0/2534222401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7181116e0 0x7fa7181a5d20 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:51.311 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.309+0000 7fa7204ac640 1 -- 172.21.15.135:0/2534222401 shutdown_connections 2023-08-24T10:04:51.311 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.309+0000 7fa7204ac640 1 --2- 172.21.15.135:0/2534222401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7181116e0 0x7fa7181a5d20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:51.311 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.309+0000 7fa7204ac640 1 -- 172.21.15.135:0/2534222401 >> 172.21.15.135:0/2534222401 conn(0x7fa71810d290 msgr2=0x7fa71810f990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:04:51.311 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.309+0000 7fa7204ac640 1 -- 172.21.15.135:0/2534222401 shutdown_connections 2023-08-24T10:04:51.311 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.309+0000 7fa7204ac640 1 -- 172.21.15.135:0/2534222401 wait complete. 2023-08-24T10:04:51.321 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd new 9a042bd6-2e19-486f-a3df-91d939324042 3 2023-08-24T10:04:51.366 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:51.364+0000 7fe109564f00 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-24T10:04:51.415 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:51.412+0000 7fe109564f00 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-24T10:04:51.425 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.419+0000 7f01072a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:51.426 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.424+0000 7f01072a3640 1 Processor -- start 2023-08-24T10:04:51.426 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.424+0000 7f01072a3640 1 -- start start 2023-08-24T10:04:51.428 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.425+0000 7f01072a3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f010011a3a0 0x7f010011c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:51.429 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.425+0000 7f01072a3640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010011cd90 con 0x7f01001116e0 2023-08-24T10:04:51.429 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.425+0000 7f01072a3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010011df30 con 0x7f010011a3a0 2023-08-24T10:04:51.429 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.425+0000 7f010483e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f010011a3a0 0x7f010011c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:51.429 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.425+0000 7f010483e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f010011a3a0 0x7f010011c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.135:43964/0 (socket says 172.21.15.135:43964) 2023-08-24T10:04:51.429 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.425+0000 7f010483e640 1 -- 172.21.15.135:0/4087643970 learned_addr learned my addr 172.21.15.135:0/4087643970 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-24T10:04:51.429 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.425+0000 7f010503f640 1 --1- 172.21.15.135:0/4087643970 >> v1:172.21.15.60:6789/0 conn(0x7f01001116e0 0x7f0100111af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.135:44016/0 (socket says 172.21.15.135:44016) 2023-08-24T10:04:51.429 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.425+0000 7f010483e640 1 -- 172.21.15.135:0/4087643970 >> v1:172.21.15.60:6789/0 conn(0x7f01001116e0 legacy=0x7f0100111af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:51.430 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.425+0000 7f010483e640 1 -- 172.21.15.135:0/4087643970 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010011e070 con 0x7f010011a3a0 2023-08-24T10:04:51.430 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.426+0000 7f010483e640 1 --2- 172.21.15.135:0/4087643970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f010011a3a0 0x7f010011c7c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1f5ef85aad6970 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:51.430 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.426+0000 7f00f7fff640 1 -- 172.21.15.135:0/4087643970 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f00f0009dd0 con 0x7f010011a3a0 2023-08-24T10:04:51.430 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.426+0000 7f00f7fff640 1 -- 172.21.15.135:0/4087643970 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f00f000e230 con 0x7f010011a3a0 2023-08-24T10:04:51.430 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.426+0000 7f00f7fff640 1 -- 172.21.15.135:0/4087643970 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f00f000e4d0 con 0x7f010011a3a0 2023-08-24T10:04:51.430 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.427+0000 7f01072a3640 1 -- 172.21.15.135:0/4087643970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f010011a3a0 msgr2=0x7f010011c7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:51.430 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.427+0000 7f01072a3640 1 --2- 172.21.15.135:0/4087643970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f010011a3a0 0x7f010011c7c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:51.431 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.427+0000 7f01072a3640 1 -- 172.21.15.135:0/4087643970 shutdown_connections 2023-08-24T10:04:51.431 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.427+0000 7f01072a3640 1 --2- 172.21.15.135:0/4087643970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f010011a3a0 0x7f010011c7c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:51.431 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.427+0000 7f01072a3640 1 -- 172.21.15.135:0/4087643970 >> 172.21.15.135:0/4087643970 conn(0x7f010010d290 msgr2=0x7f010010f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:04:51.431 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.427+0000 7f01072a3640 1 -- 172.21.15.135:0/4087643970 shutdown_connections 2023-08-24T10:04:51.431 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.427+0000 7f01072a3640 1 -- 172.21.15.135:0/4087643970 wait complete. 2023-08-24T10:04:51.431 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.427+0000 7f01072a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:51.431 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.427+0000 7f01072a3640 1 Processor -- start 2023-08-24T10:04:51.431 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.427+0000 7f01072a3640 1 -- start start 2023-08-24T10:04:51.434 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.428+0000 7f01072a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01001116e0 0x7f01001a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:51.434 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.428+0000 7f01072a3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001a6380 con 0x7f01001116e0 2023-08-24T10:04:51.434 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.428+0000 7f010503f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01001116e0 0x7f01001a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:51.434 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.428+0000 7f010503f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01001116e0 0x7f01001a5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.135:43976/0 (socket says 172.21.15.135:43976) 2023-08-24T10:04:51.435 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.428+0000 7f010503f640 1 -- 172.21.15.135:0/986176231 learned_addr learned my addr 172.21.15.135:0/986176231 (peer_addr_for_me v2:172.21.15.135:0/0) 2023-08-24T10:04:51.435 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.428+0000 7f010503f640 1 -- 172.21.15.135:0/986176231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01001a64c0 con 0x7f01001116e0 2023-08-24T10:04:51.435 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.429+0000 7f010503f640 1 --2- 172.21.15.135:0/986176231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01001116e0 0x7f01001a5db0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:51.435 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.429+0000 7f00f5ffb640 1 -- 172.21.15.135:0/986176231 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f00fc0071e0 con 0x7f01001116e0 2023-08-24T10:04:51.435 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.429+0000 7f00f5ffb640 1 -- 172.21.15.135:0/986176231 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f00fc007340 con 0x7f01001116e0 2023-08-24T10:04:51.435 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.429+0000 7f01072a3640 1 -- 172.21.15.135:0/986176231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01001a6770 con 0x7f01001116e0 2023-08-24T10:04:51.435 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.429+0000 7f00f5ffb640 1 -- 172.21.15.135:0/986176231 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f00fc0075e0 con 0x7f01001116e0 2023-08-24T10:04:51.435 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.429+0000 7f01072a3640 1 -- 172.21.15.135:0/986176231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01001aa630 con 0x7f01001116e0 2023-08-24T10:04:51.436 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.429+0000 7f00f5ffb640 1 -- 172.21.15.135:0/986176231 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (crc 0 0 0) 0x7f00fc005ae0 con 0x7f01001116e0 2023-08-24T10:04:51.436 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.429+0000 7f00f5ffb640 1 -- 172.21.15.135:0/986176231 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1092+0+0 (crc 0 0 0) 0x7f00fc005d90 con 0x7f01001116e0 2023-08-24T10:04:51.436 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.430+0000 7f01072a3640 1 -- 172.21.15.135:0/986176231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01001a6960 con 0x7f01001116e0 2023-08-24T10:04:51.436 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.432+0000 7f00f5ffb640 1 -- 172.21.15.135:0/986176231 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (crc 0 0 0) 0x7f01001a6960 con 0x7f01001116e0 2023-08-24T10:04:51.511 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.501+0000 7f01072a3640 1 -- 172.21.15.135:0/986176231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9a042bd6-2e19-486f-a3df-91d939324042", "id": 3} v 0) v1 -- 0x7f010011a3a0 con 0x7f01001116e0 2023-08-24T10:04:51.512 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.503+0000 7f00f5ffb640 1 -- 172.21.15.135:0/986176231 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9a042bd6-2e19-486f-a3df-91d939324042", "id": 3}]=0 v7) v1 ==== 112+0+2 (crc 0 0 0) 0x7f00fc00c880 con 0x7f01001116e0 2023-08-24T10:04:51.512 INFO:teuthology.orchestra.run.smithi135.stdout:3 2023-08-24T10:04:51.513 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.505+0000 7f01072a3640 1 -- 172.21.15.135:0/986176231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01001116e0 msgr2=0x7f01001a5db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:51.513 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.505+0000 7f01072a3640 1 --2- 172.21.15.135:0/986176231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01001116e0 0x7f01001a5db0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:51.513 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.505+0000 7f01072a3640 1 -- 172.21.15.135:0/986176231 shutdown_connections 2023-08-24T10:04:51.513 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.505+0000 7f01072a3640 1 --2- 172.21.15.135:0/986176231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01001116e0 0x7f01001a5db0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:51.514 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.505+0000 7f01072a3640 1 -- 172.21.15.135:0/986176231 >> 172.21.15.135:0/986176231 conn(0x7f010010d290 msgr2=0x7f0100113040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:04:51.514 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.505+0000 7f01072a3640 1 -- 172.21.15.135:0/986176231 shutdown_connections 2023-08-24T10:04:51.514 INFO:teuthology.orchestra.run.smithi135.stderr:2023-08-24T10:04:51.505+0000 7f01072a3640 1 -- 172.21.15.135:0/986176231 wait complete. 2023-08-24T10:04:51.516 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2023-08-24T10:04:51.525 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:51.524+0000 7fe109564f00 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-24T10:04:51.559 INFO:tasks.ceph.osd.0:Restarting daemon 2023-08-24T10:04:51.560 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2023-08-24T10:04:51.603 INFO:tasks.ceph.osd.0:Started 2023-08-24T10:04:51.603 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2023-08-24T10:04:51.645 INFO:tasks.ceph.osd.1:Restarting daemon 2023-08-24T10:04:51.645 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2023-08-24T10:04:51.661 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:51.659+0000 7fe109564f00 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-24T10:04:51.687 INFO:tasks.ceph.osd.1:Started 2023-08-24T10:04:51.688 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2023-08-24T10:04:51.720 INFO:tasks.ceph.osd.2:Restarting daemon 2023-08-24T10:04:51.721 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2023-08-24T10:04:51.760 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:51.758+0000 7fe109564f00 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-24T10:04:51.762 INFO:tasks.ceph.osd.2:Started 2023-08-24T10:04:51.762 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2023-08-24T10:04:51.794 INFO:tasks.ceph.osd.3:Restarting daemon 2023-08-24T10:04:51.795 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2023-08-24T10:04:51.806 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:51.804+0000 7fe109564f00 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-24T10:04:51.808 INFO:tasks.ceph.osd.3:Started 2023-08-24T10:04:51.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-24T10:04:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.932+0000 7fdd3cb9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:51.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.937+0000 7fdd3cb9e640 1 Processor -- start 2023-08-24T10:04:51.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.937+0000 7fdd3cb9e640 1 -- start start 2023-08-24T10:04:51.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.938+0000 7fdd3cb9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdd38133140 0x7fdd38137970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:51.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.938+0000 7fdd3cb9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd38133550 con 0x7fdd3809c4a0 2023-08-24T10:04:51.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.938+0000 7fdd3cb9e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd3809d8d0 con 0x7fdd38133140 2023-08-24T10:04:51.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.938+0000 7fdd2ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdd38133140 0x7fdd38137970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:51.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.938+0000 7fdd3659c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdd3809c4a0 0x7fdd38132a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49040/0 (socket says 172.21.15.60:49040) 2023-08-24T10:04:51.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.938+0000 7fdd3659c640 1 -- 172.21.15.60:0/84699564 learned_addr learned my addr 172.21.15.60:0/84699564 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:04:51.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.939+0000 7fdd2ffff640 1 -- 172.21.15.60:0/84699564 >> v1:172.21.15.60:6789/0 conn(0x7fdd3809c4a0 legacy=0x7fdd38132a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:51.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.939+0000 7fdd2ffff640 1 -- 172.21.15.60:0/84699564 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd38132720 con 0x7fdd38133140 2023-08-24T10:04:51.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.939+0000 7fdd2ffff640 1 --2- 172.21.15.60:0/84699564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd38133140 0x7fdd38137970 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951a7f9e28f92228 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:51.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.939+0000 7fdd35d9b640 1 -- 172.21.15.60:0/84699564 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdd20009ce0 con 0x7fdd38133140 2023-08-24T10:04:51.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.939+0000 7fdd35d9b640 1 -- 172.21.15.60:0/84699564 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fdd20009e40 con 0x7fdd38133140 2023-08-24T10:04:51.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.939+0000 7fdd35d9b640 1 -- 172.21.15.60:0/84699564 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdd2000e360 con 0x7fdd38133140 2023-08-24T10:04:51.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.940+0000 7fdd3cb9e640 1 -- 172.21.15.60:0/84699564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd38133140 msgr2=0x7fdd38137970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:51.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.940+0000 7fdd3cb9e640 1 --2- 172.21.15.60:0/84699564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd38133140 0x7fdd38137970 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:51.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.940+0000 7fdd3cb9e640 1 -- 172.21.15.60:0/84699564 shutdown_connections 2023-08-24T10:04:51.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.940+0000 7fdd3cb9e640 1 --2- 172.21.15.60:0/84699564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd38133140 0x7fdd38137970 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:51.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.940+0000 7fdd3cb9e640 1 -- 172.21.15.60:0/84699564 >> 172.21.15.60:0/84699564 conn(0x7fdd380a2500 msgr2=0x7fdd380a2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:04:51.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.940+0000 7fdd3cb9e640 1 -- 172.21.15.60:0/84699564 shutdown_connections 2023-08-24T10:04:51.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.940+0000 7fdd3cb9e640 1 -- 172.21.15.60:0/84699564 wait complete. 2023-08-24T10:04:51.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.940+0000 7fdd3cb9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:51.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.942+0000 7fdd3cb9e640 1 Processor -- start 2023-08-24T10:04:51.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.942+0000 7fdd3cb9e640 1 -- start start 2023-08-24T10:04:51.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.942+0000 7fdd3cb9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd3809c4a0 0x7fdd381c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:51.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.942+0000 7fdd3cb9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd381c4ce0 con 0x7fdd3809c4a0 2023-08-24T10:04:51.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.943+0000 7fdd3659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd3809c4a0 0x7fdd381c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:51.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.943+0000 7fdd3659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd3809c4a0 0x7fdd381c6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53664/0 (socket says 172.21.15.60:53664) 2023-08-24T10:04:51.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.943+0000 7fdd3659c640 1 -- 172.21.15.60:0/3367194721 learned_addr learned my addr 172.21.15.60:0/3367194721 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:04:51.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.943+0000 7fdd3659c640 1 -- 172.21.15.60:0/3367194721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3811c580 con 0x7fdd3809c4a0 2023-08-24T10:04:51.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.943+0000 7fdd3659c640 1 --2- 172.21.15.60:0/3367194721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd3809c4a0 0x7fdd381c6f50 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:51.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.943+0000 7fdd2effd640 1 -- 172.21.15.60:0/3367194721 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdd280059d0 con 0x7fdd3809c4a0 2023-08-24T10:04:51.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.943+0000 7fdd2effd640 1 -- 172.21.15.60:0/3367194721 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fdd28005b30 con 0x7fdd3809c4a0 2023-08-24T10:04:51.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.943+0000 7fdd3cb9e640 1 -- 172.21.15.60:0/3367194721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd380a1990 con 0x7fdd3809c4a0 2023-08-24T10:04:51.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.943+0000 7fdd2effd640 1 -- 172.21.15.60:0/3367194721 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdd28005dd0 con 0x7fdd3809c4a0 2023-08-24T10:04:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.944+0000 7fdd3cb9e640 1 -- 172.21.15.60:0/3367194721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd3811c1a0 con 0x7fdd3809c4a0 2023-08-24T10:04:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.944+0000 7fdd2effd640 1 -- 172.21.15.60:0/3367194721 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (crc 0 0 0) 0x7fdd280073b0 con 0x7fdd3809c4a0 2023-08-24T10:04:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.944+0000 7fdd2effd640 1 -- 172.21.15.60:0/3367194721 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1207+0+0 (crc 0 0 0) 0x7fdd2800f750 con 0x7fdd3809c4a0 2023-08-24T10:04:51.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.945+0000 7fdd3cb9e640 1 -- 172.21.15.60:0/3367194721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd38061800 con 0x7fdd3809c4a0 2023-08-24T10:04:51.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:51.947+0000 7fdd2effd640 1 -- 172.21.15.60:0/3367194721 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (crc 0 0 0) 0x7fdd2800f230 con 0x7fdd3809c4a0 2023-08-24T10:04:51.957 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:51.956+0000 7fe109564f00 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-24T10:04:52.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:52.011+0000 7fdd3cb9e640 1 -- 172.21.15.60:0/3367194721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd380735e0 con 0x7fdd3809c4a0 2023-08-24T10:04:52.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:52.011+0000 7fdd2effd640 1 -- 172.21.15.60:0/3367194721 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) v1 ==== 74+0+4151 (crc 0 0 0) 0x7fdd2800f480 con 0x7fdd3809c4a0 2023-08-24T10:04:52.013 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:04:52.013 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":7,"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","created":"2023-08-24T10:04:49.373034+0000","modified":"2023-08-24T10:04:51.502866+0000","last_up_change":"0.000000","last_in_change":"2023-08-24T10:04:51.502866+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"72632e8e-7a21-476f-91d4-c34b43117b99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3157bd05-35e6-4d2c-9e8b-ec16937c5e5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"77ec2ee0-440e-41ae-ab0b-0427531dfa8a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9a042bd6-2e19-486f-a3df-91d939324042","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-24T10:04:52.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:52.012+0000 7fdd3cb9e640 1 -- 172.21.15.60:0/3367194721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd3809c4a0 msgr2=0x7fdd381c6f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:52.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:52.012+0000 7fdd3cb9e640 1 --2- 172.21.15.60:0/3367194721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd3809c4a0 0x7fdd381c6f50 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:52.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:52.012+0000 7fdd3cb9e640 1 -- 172.21.15.60:0/3367194721 shutdown_connections 2023-08-24T10:04:52.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:52.012+0000 7fdd3cb9e640 1 --2- 172.21.15.60:0/3367194721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd3809c4a0 0x7fdd381c6f50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:52.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:52.012+0000 7fdd3cb9e640 1 -- 172.21.15.60:0/3367194721 >> 172.21.15.60:0/3367194721 conn(0x7fdd380a2500 msgr2=0x7fdd381ba670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:04:52.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:52.012+0000 7fdd3cb9e640 1 -- 172.21.15.60:0/3367194721 shutdown_connections 2023-08-24T10:04:52.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:52.013+0000 7fdd3cb9e640 1 -- 172.21.15.60:0/3367194721 wait complete. 2023-08-24T10:04:52.028 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-08-24T10:04:52.029 INFO:tasks.ceph:Waiting for OSDs to come up 2023-08-24T10:04:52.425 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:52.423+0000 7fe109564f00 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-24T10:04:52.433 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-08-24T10:04:52.533 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.528+0000 7f32da4ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:52.533 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.532+0000 7f32da4ec640 1 Processor -- start 2023-08-24T10:04:52.534 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.532+0000 7f32da4ec640 1 -- start start 2023-08-24T10:04:52.535 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.533+0000 7f32da4ec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f32d4133190 0x7f32d409c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:52.535 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.533+0000 7f32da4ec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d409c810 con 0x7f32d409b3f0 2023-08-24T10:04:52.536 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.533+0000 7f32da4ec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d41335a0 con 0x7f32d4133190 2023-08-24T10:04:52.536 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.533+0000 7f32d37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f32d4133190 0x7f32d409c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:52.536 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.533+0000 7f32d3fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f32d409b3f0 0x7f32d4132a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49052/0 (socket says 172.21.15.60:49052) 2023-08-24T10:04:52.537 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.533+0000 7f32d3fff640 1 -- 172.21.15.60:0/2158111330 learned_addr learned my addr 172.21.15.60:0/2158111330 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:04:52.537 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.534+0000 7f32d37fe640 1 -- 172.21.15.60:0/2158111330 >> v1:172.21.15.60:6789/0 conn(0x7f32d409b3f0 legacy=0x7f32d4132a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:52.537 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.534+0000 7f32d37fe640 1 -- 172.21.15.60:0/2158111330 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d409ab90 con 0x7f32d4133190 2023-08-24T10:04:52.537 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.534+0000 7f32d2ffd640 1 -- 172.21.15.60:0/2158111330 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 832681977 0 0) 0x7f32d409c810 con 0x7f32d409b3f0 2023-08-24T10:04:52.538 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.534+0000 7f32d37fe640 1 --2- 172.21.15.60:0/2158111330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d4133190 0x7f32d409c080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed155acc75b0ebe server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:52.538 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.534+0000 7f32d2ffd640 1 -- 172.21.15.60:0/2158111330 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f32c4009ce0 con 0x7f32d4133190 2023-08-24T10:04:52.538 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.534+0000 7f32d2ffd640 1 -- 172.21.15.60:0/2158111330 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f32c4009e40 con 0x7f32d4133190 2023-08-24T10:04:52.538 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.534+0000 7f32d2ffd640 1 -- 172.21.15.60:0/2158111330 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f32c400e360 con 0x7f32d4133190 2023-08-24T10:04:52.542 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.535+0000 7f32da4ec640 1 -- 172.21.15.60:0/2158111330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d4133190 msgr2=0x7f32d409c080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:52.542 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.535+0000 7f32da4ec640 1 --2- 172.21.15.60:0/2158111330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d4133190 0x7f32d409c080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:52.542 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.535+0000 7f32da4ec640 1 -- 172.21.15.60:0/2158111330 shutdown_connections 2023-08-24T10:04:52.542 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.535+0000 7f32da4ec640 1 --2- 172.21.15.60:0/2158111330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d4133190 0x7f32d409c080 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:52.543 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.535+0000 7f32da4ec640 1 -- 172.21.15.60:0/2158111330 >> 172.21.15.60:0/2158111330 conn(0x7f32d40a11e0 msgr2=0x7f32d40a15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:04:52.543 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.535+0000 7f32da4ec640 1 -- 172.21.15.60:0/2158111330 shutdown_connections 2023-08-24T10:04:52.543 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.535+0000 7f32da4ec640 1 -- 172.21.15.60:0/2158111330 wait complete. 2023-08-24T10:04:52.543 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.535+0000 7f32da4ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:52.543 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.536+0000 7f32da4ec640 1 Processor -- start 2023-08-24T10:04:52.543 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.536+0000 7f32da4ec640 1 -- start start 2023-08-24T10:04:52.543 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.537+0000 7f32da4ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d4133190 0x7f32d411ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:52.544 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.537+0000 7f32da4ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d412f9e0 con 0x7f32d4133190 2023-08-24T10:04:52.544 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.537+0000 7f32d3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d4133190 0x7f32d411ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:52.544 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.537+0000 7f32d3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d4133190 0x7f32d411ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53668/0 (socket says 172.21.15.60:53668) 2023-08-24T10:04:52.544 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.537+0000 7f32d3fff640 1 -- 172.21.15.60:0/2395300797 learned_addr learned my addr 172.21.15.60:0/2395300797 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:04:52.544 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.537+0000 7f32d3fff640 1 -- 172.21.15.60:0/2395300797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d412f7c0 con 0x7f32d4133190 2023-08-24T10:04:52.545 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.538+0000 7f32d3fff640 1 --2- 172.21.15.60:0/2395300797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d4133190 0x7f32d411ad30 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:52.545 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.538+0000 7f32d0ff9640 1 -- 172.21.15.60:0/2395300797 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f32bc004af0 con 0x7f32d4133190 2023-08-24T10:04:52.545 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.538+0000 7f32d0ff9640 1 -- 172.21.15.60:0/2395300797 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f32bc004c50 con 0x7f32d4133190 2023-08-24T10:04:52.546 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.538+0000 7f32da4ec640 1 -- 172.21.15.60:0/2395300797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d406a310 con 0x7f32d4133190 2023-08-24T10:04:52.546 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.538+0000 7f32d0ff9640 1 -- 172.21.15.60:0/2395300797 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f32bc007120 con 0x7f32d4133190 2023-08-24T10:04:52.546 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.538+0000 7f32da4ec640 1 -- 172.21.15.60:0/2395300797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d411c550 con 0x7f32d4133190 2023-08-24T10:04:52.547 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.538+0000 7f32d0ff9640 1 -- 172.21.15.60:0/2395300797 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (crc 0 0 0) 0x7f32bc00c570 con 0x7f32d4133190 2023-08-24T10:04:52.547 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.538+0000 7f32d0ff9640 1 -- 172.21.15.60:0/2395300797 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1207+0+0 (crc 0 0 0) 0x7f32bc00f7f0 con 0x7f32d4133190 2023-08-24T10:04:52.547 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.539+0000 7f32da4ec640 1 -- 172.21.15.60:0/2395300797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d4061800 con 0x7f32d4133190 2023-08-24T10:04:52.548 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.541+0000 7f32d0ff9640 1 -- 172.21.15.60:0/2395300797 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (crc 0 0 0) 0x7f32bc00f2d0 con 0x7f32d4133190 2023-08-24T10:04:52.604 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.603+0000 7f32da4ec640 1 -- 172.21.15.60:0/2395300797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32d4073680 con 0x7f32d4133190 2023-08-24T10:04:52.605 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.604+0000 7f32d0ff9640 1 -- 172.21.15.60:0/2395300797 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) v1 ==== 74+0+4151 (crc 0 0 0) 0x7f32bc00f470 con 0x7f32d4133190 2023-08-24T10:04:52.606 INFO:teuthology.misc.health.smithi060.stdout: 2023-08-24T10:04:52.606 INFO:teuthology.misc.health.smithi060.stdout:{"epoch":7,"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","created":"2023-08-24T10:04:49.373034+0000","modified":"2023-08-24T10:04:51.502866+0000","last_up_change":"0.000000","last_in_change":"2023-08-24T10:04:51.502866+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"72632e8e-7a21-476f-91d4-c34b43117b99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3157bd05-35e6-4d2c-9e8b-ec16937c5e5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"77ec2ee0-440e-41ae-ab0b-0427531dfa8a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9a042bd6-2e19-486f-a3df-91d939324042","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-24T10:04:52.607 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.605+0000 7f32da4ec640 1 -- 172.21.15.60:0/2395300797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d4133190 msgr2=0x7f32d411ad30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:52.607 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.605+0000 7f32da4ec640 1 --2- 172.21.15.60:0/2395300797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d4133190 0x7f32d411ad30 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:52.607 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.605+0000 7f32da4ec640 1 -- 172.21.15.60:0/2395300797 shutdown_connections 2023-08-24T10:04:52.607 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.605+0000 7f32da4ec640 1 --2- 172.21.15.60:0/2395300797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d4133190 0x7f32d411ad30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:52.607 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.605+0000 7f32da4ec640 1 -- 172.21.15.60:0/2395300797 >> 172.21.15.60:0/2395300797 conn(0x7f32d40a11e0 msgr2=0x7f32d41303f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:04:52.607 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.605+0000 7f32da4ec640 1 -- 172.21.15.60:0/2395300797 shutdown_connections 2023-08-24T10:04:52.608 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:52.605+0000 7f32da4ec640 1 -- 172.21.15.60:0/2395300797 wait complete. 2023-08-24T10:04:52.615 DEBUG:teuthology.misc:0 of 4 OSDs are up 2023-08-24T10:04:52.634 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:52.633+0000 7fe109564f00 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-24T10:04:52.691 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:52.689+0000 7fe109564f00 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-24T10:04:52.740 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:52.738+0000 7fe109564f00 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-24T10:04:52.796 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:52.795+0000 7fe109564f00 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-24T10:04:53.081 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:53.080+0000 7fe109564f00 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-24T10:04:53.135 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:53.133+0000 7fe109564f00 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-24T10:04:53.183 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:53.182+0000 7fe109564f00 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-24T10:04:53.331 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:53.329+0000 7fe109564f00 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-24T10:04:53.374 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:53.373+0000 7fe109564f00 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-24T10:04:53.431 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:53.427+0000 7fe109564f00 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-24T10:04:53.479 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:53.478+0000 7fe109564f00 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-24T10:04:54.499 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:04:54.498+0000 7fe0a9234640 -1 mgr.server handle_report got status from non-daemon mon.a 2023-08-24T10:04:59.020 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-08-24T10:04:59.118 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.113+0000 7f723439e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:59.119 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.117+0000 7f723439e640 1 Processor -- start 2023-08-24T10:04:59.119 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.117+0000 7f723439e640 1 -- start start 2023-08-24T10:04:59.121 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.118+0000 7f723439e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f722c132a10 0x7f722c134e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:59.121 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.118+0000 7f723439e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f722c06d030 con 0x7f722c12f930 2023-08-24T10:04:59.122 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.118+0000 7f723439e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f722c071d70 con 0x7f722c132a10 2023-08-24T10:04:59.122 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.118+0000 7f7231939640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f722c132a10 0x7f722c134e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:59.122 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.118+0000 7f723213a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f722c12f930 0x7f722c12fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52188/0 (socket says 172.21.15.60:52188) 2023-08-24T10:04:59.122 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.118+0000 7f723213a640 1 -- 172.21.15.60:0/4024717348 learned_addr learned my addr 172.21.15.60:0/4024717348 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:04:59.123 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.119+0000 7f7231939640 1 -- 172.21.15.60:0/4024717348 >> v1:172.21.15.60:6789/0 conn(0x7f722c12f930 legacy=0x7f722c12fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:59.123 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.119+0000 7f7231939640 1 -- 172.21.15.60:0/4024717348 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c12f700 con 0x7f722c132a10 2023-08-24T10:04:59.123 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.119+0000 7f7231939640 1 --2- 172.21.15.60:0/4024717348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f722c132a10 0x7f722c134e30 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dceb074220c6878 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:59.187 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.185+0000 7f7231138640 1 -- 172.21.15.60:0/4024717348 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7228009ce0 con 0x7f722c132a10 2023-08-24T10:04:59.187 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.185+0000 7f7231138640 1 -- 172.21.15.60:0/4024717348 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7228009e40 con 0x7f722c132a10 2023-08-24T10:04:59.188 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.185+0000 7f7231138640 1 -- 172.21.15.60:0/4024717348 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f722800e360 con 0x7f722c132a10 2023-08-24T10:04:59.188 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.186+0000 7f723439e640 1 -- 172.21.15.60:0/4024717348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f722c132a10 msgr2=0x7f722c134e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:59.188 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.186+0000 7f723439e640 1 --2- 172.21.15.60:0/4024717348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f722c132a10 0x7f722c134e30 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:59.189 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.186+0000 7f723439e640 1 -- 172.21.15.60:0/4024717348 shutdown_connections 2023-08-24T10:04:59.189 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.186+0000 7f723439e640 1 --2- 172.21.15.60:0/4024717348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f722c132a10 0x7f722c134e30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:59.189 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.186+0000 7f723439e640 1 -- 172.21.15.60:0/4024717348 >> 172.21.15.60:0/4024717348 conn(0x7f722c09e1f0 msgr2=0x7f722c09e600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:04:59.189 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.186+0000 7f723439e640 1 -- 172.21.15.60:0/4024717348 shutdown_connections 2023-08-24T10:04:59.190 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.186+0000 7f723439e640 1 -- 172.21.15.60:0/4024717348 wait complete. 2023-08-24T10:04:59.190 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.186+0000 7f723439e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:59.193 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.187+0000 7f723439e640 1 Processor -- start 2023-08-24T10:04:59.194 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.187+0000 7f723439e640 1 -- start start 2023-08-24T10:04:59.194 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.188+0000 7f723439e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f722c12f930 0x7f722c1270a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:59.194 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.188+0000 7f723439e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c1281a0 con 0x7f722c12f930 2023-08-24T10:04:59.194 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.188+0000 7f723213a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f722c12f930 0x7f722c1270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:59.195 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.188+0000 7f723213a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f722c12f930 0x7f722c1270a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32950/0 (socket says 172.21.15.60:32950) 2023-08-24T10:04:59.195 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.188+0000 7f723213a640 1 -- 172.21.15.60:0/2507876778 learned_addr learned my addr 172.21.15.60:0/2507876778 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:04:59.195 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.188+0000 7f723213a640 1 -- 172.21.15.60:0/2507876778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c1268d0 con 0x7f722c12f930 2023-08-24T10:04:59.195 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.188+0000 7f723213a640 1 --2- 172.21.15.60:0/2507876778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f722c12f930 0x7f722c1270a0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:59.196 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.188+0000 7f7222ffd640 1 -- 172.21.15.60:0/2507876778 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f721c0071e0 con 0x7f722c12f930 2023-08-24T10:04:59.196 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.189+0000 7f7222ffd640 1 -- 172.21.15.60:0/2507876778 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f721c007340 con 0x7f722c12f930 2023-08-24T10:04:59.196 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.189+0000 7f7222ffd640 1 -- 172.21.15.60:0/2507876778 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f721c0075e0 con 0x7f722c12f930 2023-08-24T10:04:59.197 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.189+0000 7f723439e640 1 -- 172.21.15.60:0/2507876778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f722c126b70 con 0x7f722c12f930 2023-08-24T10:04:59.197 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.189+0000 7f723439e640 1 -- 172.21.15.60:0/2507876778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722c128640 con 0x7f722c12f930 2023-08-24T10:04:59.197 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.189+0000 7f723439e640 1 -- 172.21.15.60:0/2507876778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f722c061800 con 0x7f722c12f930 2023-08-24T10:04:59.197 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.190+0000 7f7222ffd640 1 -- 172.21.15.60:0/2507876778 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) v1 ==== 44355+0+0 (crc 0 0 0) 0x7f721c005770 con 0x7f722c12f930 2023-08-24T10:04:59.198 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.190+0000 7f7222ffd640 1 --2- 172.21.15.60:0/2507876778 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f72100370a0 0x7f72100395b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:59.198 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.190+0000 7f7222ffd640 1 -- 172.21.15.60:0/2507876778 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2319+0+0 (crc 0 0 0) 0x7f721c0460b0 con 0x7f722c12f930 2023-08-24T10:04:59.198 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.190+0000 7f7231939640 1 --2- 172.21.15.60:0/2507876778 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f72100370a0 0x7f72100395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:59.198 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.190+0000 7f7231939640 1 --2- 172.21.15.60:0/2507876778 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f72100370a0 0x7f72100395b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:59.199 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.195+0000 7f7222ffd640 1 -- 172.21.15.60:0/2507876778 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f721c00d330 con 0x7f722c12f930 2023-08-24T10:04:59.378 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.377+0000 7f723439e640 1 -- 172.21.15.60:0/2507876778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f722c071920 con 0x7f722c12f930 2023-08-24T10:04:59.379 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.377+0000 7f7222ffd640 1 -- 172.21.15.60:0/2507876778 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+6913 (crc 0 0 0) 0x7f721c00d150 con 0x7f722c12f930 2023-08-24T10:04:59.379 INFO:teuthology.misc.health.smithi060.stdout: 2023-08-24T10:04:59.380 INFO:teuthology.misc.health.smithi060.stdout:{"epoch":11,"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","created":"2023-08-24T10:04:49.373034+0000","modified":"2023-08-24T10:04:58.609181+0000","last_up_change":"2023-08-24T10:04:55.538474+0000","last_in_change":"2023-08-24T10:04:51.502866+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-24T10:04:57.553657+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72632e8e-7a21-476f-91d4-c34b43117b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2841298624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":2841298624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2841298624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":2841298624}]},"public_addr":"172.21.15.60:6800/2841298624","cluster_addr":"172.21.15.60:6801/2841298624","heartbeat_back_addr":"172.21.15.60:6804/2841298624","heartbeat_front_addr":"172.21.15.60:6805/2841298624","state":["exists","up"]},{"osd":1,"uuid":"3157bd05-35e6-4d2c-9e8b-ec16937c5e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2103781563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6803","nonce":2103781563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6807","nonce":2103781563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2103781563}]},"public_addr":"172.21.15.60:6802/2103781563","cluster_addr":"172.21.15.60:6803/2103781563","heartbeat_back_addr":"172.21.15.60:6807/2103781563","heartbeat_front_addr":"172.21.15.60:6806/2103781563","state":["exists","up"]},{"osd":2,"uuid":"77ec2ee0-440e-41ae-ab0b-0427531dfa8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1298459676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6805","nonce":1298459676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1298459676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":1298459676}]},"public_addr":"172.21.15.135:6804/1298459676","cluster_addr":"172.21.15.135:6805/1298459676","heartbeat_back_addr":"172.21.15.135:6808/1298459676","heartbeat_front_addr":"172.21.15.135:6809/1298459676","state":["exists","up"]},{"osd":3,"uuid":"9a042bd6-2e19-486f-a3df-91d939324042","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3144688060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6803","nonce":3144688060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6807","nonce":3144688060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3144688060}]},"public_addr":"172.21.15.135:6802/3144688060","cluster_addr":"172.21.15.135:6803/3144688060","heartbeat_back_addr":"172.21.15.135:6807/3144688060","heartbeat_front_addr":"172.21.15.135:6806/3144688060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-24T10:04:59.381 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.379+0000 7f723439e640 1 -- 172.21.15.60:0/2507876778 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f72100370a0 msgr2=0x7f72100395b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:59.382 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.379+0000 7f723439e640 1 --2- 172.21.15.60:0/2507876778 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f72100370a0 0x7f72100395b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:59.382 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.379+0000 7f723439e640 1 -- 172.21.15.60:0/2507876778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f722c12f930 msgr2=0x7f722c1270a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:59.382 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.379+0000 7f723439e640 1 --2- 172.21.15.60:0/2507876778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f722c12f930 0x7f722c1270a0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:59.382 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.379+0000 7f723439e640 1 -- 172.21.15.60:0/2507876778 shutdown_connections 2023-08-24T10:04:59.383 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.379+0000 7f723439e640 1 --2- 172.21.15.60:0/2507876778 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f72100370a0 0x7f72100395b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:59.383 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.379+0000 7f723439e640 1 --2- 172.21.15.60:0/2507876778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f722c12f930 0x7f722c1270a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:59.383 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.379+0000 7f723439e640 1 -- 172.21.15.60:0/2507876778 >> 172.21.15.60:0/2507876778 conn(0x7f722c09e1f0 msgr2=0x7f722c12e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:04:59.383 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.379+0000 7f723439e640 1 -- 172.21.15.60:0/2507876778 shutdown_connections 2023-08-24T10:04:59.384 INFO:teuthology.misc.health.smithi060.stderr:2023-08-24T10:04:59.379+0000 7f723439e640 1 -- 172.21.15.60:0/2507876778 wait complete. 2023-08-24T10:04:59.390 DEBUG:teuthology.misc:4 of 4 OSDs are up 2023-08-24T10:04:59.391 INFO:tasks.ceph:Creating RBD pool 2023-08-24T10:04:59.391 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph osd pool create rbd 8 2023-08-24T10:04:59.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.496+0000 7f5ae2663640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:59.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.502+0000 7f5ae2663640 1 Processor -- start 2023-08-24T10:04:59.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.503+0000 7f5ae2663640 1 -- start start 2023-08-24T10:04:59.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.503+0000 7f5ae2663640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5adc12fd20 0x7f5adc1379a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:59.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.503+0000 7f5ae2663640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5adc09c280 con 0x7f5adc09cb20 2023-08-24T10:04:59.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.503+0000 7f5ae2663640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5adc130130 con 0x7f5adc12fd20 2023-08-24T10:04:59.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.503+0000 7f5adbfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5adc09cb20 0x7f5adc12f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52202/0 (socket says 172.21.15.60:52202) 2023-08-24T10:04:59.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.503+0000 7f5adbfff640 1 -- 172.21.15.60:0/694409600 learned_addr learned my addr 172.21.15.60:0/694409600 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:04:59.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.503+0000 7f5adb7fe640 1 --2- 172.21.15.60:0/694409600 >> v2:172.21.15.60:3300/0 conn(0x7f5adc12fd20 0x7f5adc1379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:59.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.504+0000 7f5adaffd640 1 -- 172.21.15.60:0/694409600 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842218882 0 0) 0x7f5adc09c280 con 0x7f5adc09cb20 2023-08-24T10:04:59.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.504+0000 7f5adaffd640 1 -- 172.21.15.60:0/694409600 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ac40034f0 con 0x7f5adc09cb20 2023-08-24T10:04:59.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.504+0000 7f5adb7fe640 1 -- 172.21.15.60:0/694409600 >> v1:172.21.15.60:6789/0 conn(0x7f5adc09cb20 legacy=0x7f5adc12f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:59.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.504+0000 7f5adb7fe640 1 -- 172.21.15.60:0/694409600 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5adc12f300 con 0x7f5adc12fd20 2023-08-24T10:04:59.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.504+0000 7f5adb7fe640 1 --2- 172.21.15.60:0/694409600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc12fd20 0x7f5adc1379a0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ccd081a9e6dcd1 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:59.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.504+0000 7f5adaffd640 1 -- 172.21.15.60:0/694409600 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5ad0009ce0 con 0x7f5adc12fd20 2023-08-24T10:04:59.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.504+0000 7f5adaffd640 1 -- 172.21.15.60:0/694409600 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5ad0009e40 con 0x7f5adc12fd20 2023-08-24T10:04:59.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.504+0000 7f5adaffd640 1 -- 172.21.15.60:0/694409600 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5ad000e360 con 0x7f5adc12fd20 2023-08-24T10:04:59.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.505+0000 7f5ae2663640 1 -- 172.21.15.60:0/694409600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc12fd20 msgr2=0x7f5adc1379a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:04:59.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.505+0000 7f5ae2663640 1 --2- 172.21.15.60:0/694409600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc12fd20 0x7f5adc1379a0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:59.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.505+0000 7f5ae2663640 1 -- 172.21.15.60:0/694409600 shutdown_connections 2023-08-24T10:04:59.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.505+0000 7f5ae2663640 1 --2- 172.21.15.60:0/694409600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc12fd20 0x7f5adc1379a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:04:59.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.505+0000 7f5ae2663640 1 -- 172.21.15.60:0/694409600 >> 172.21.15.60:0/694409600 conn(0x7f5adc0a28a0 msgr2=0x7f5adc0a2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:04:59.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.505+0000 7f5ae2663640 1 -- 172.21.15.60:0/694409600 shutdown_connections 2023-08-24T10:04:59.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.505+0000 7f5ae2663640 1 -- 172.21.15.60:0/694409600 wait complete. 2023-08-24T10:04:59.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.505+0000 7f5ae2663640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:04:59.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.506+0000 7f5ae2663640 1 Processor -- start 2023-08-24T10:04:59.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.506+0000 7f5ae2663640 1 -- start start 2023-08-24T10:04:59.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.507+0000 7f5ae2663640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc09cb20 0x7f5adc11bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:59.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.507+0000 7f5ae2663640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc11dc40 con 0x7f5adc09cb20 2023-08-24T10:04:59.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.507+0000 7f5adbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc09cb20 0x7f5adc11bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:59.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.507+0000 7f5adbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc09cb20 0x7f5adc11bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32972/0 (socket says 172.21.15.60:32972) 2023-08-24T10:04:59.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.507+0000 7f5adbfff640 1 -- 172.21.15.60:0/2168297516 learned_addr learned my addr 172.21.15.60:0/2168297516 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:04:59.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.507+0000 7f5adbfff640 1 -- 172.21.15.60:0/2168297516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5adc11bc10 con 0x7f5adc09cb20 2023-08-24T10:04:59.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.507+0000 7f5adbfff640 1 --2- 172.21.15.60:0/2168297516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc09cb20 0x7f5adc11bf10 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:59.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.507+0000 7f5ad8ff9640 1 -- 172.21.15.60:0/2168297516 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5acc0059d0 con 0x7f5adc09cb20 2023-08-24T10:04:59.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.508+0000 7f5ad8ff9640 1 -- 172.21.15.60:0/2168297516 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5acc005b30 con 0x7f5adc09cb20 2023-08-24T10:04:59.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.508+0000 7f5ae2663640 1 -- 172.21.15.60:0/2168297516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5adc11b900 con 0x7f5adc09cb20 2023-08-24T10:04:59.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.508+0000 7f5ad8ff9640 1 -- 172.21.15.60:0/2168297516 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5acc005dd0 con 0x7f5adc09cb20 2023-08-24T10:04:59.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.508+0000 7f5ae2663640 1 -- 172.21.15.60:0/2168297516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5adc11e880 con 0x7f5adc09cb20 2023-08-24T10:04:59.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.508+0000 7f5ad8ff9640 1 -- 172.21.15.60:0/2168297516 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) v1 ==== 44355+0+0 (crc 0 0 0) 0x7f5acc0070b0 con 0x7f5adc09cb20 2023-08-24T10:04:59.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.509+0000 7f5ae2663640 1 -- 172.21.15.60:0/2168297516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5adc11dd80 con 0x7f5adc09cb20 2023-08-24T10:04:59.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.509+0000 7f5ad8ff9640 1 --2- 172.21.15.60:0/2168297516 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5ac80370a0 0x7f5ac80395b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:04:59.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.509+0000 7f5ad8ff9640 1 -- 172.21.15.60:0/2168297516 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2319+0+0 (crc 0 0 0) 0x7f5acc046130 con 0x7f5adc09cb20 2023-08-24T10:04:59.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.509+0000 7f5adb7fe640 1 --2- 172.21.15.60:0/2168297516 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5ac80370a0 0x7f5ac80395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:04:59.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.510+0000 7f5adb7fe640 1 --2- 172.21.15.60:0/2168297516 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5ac80370a0 0x7f5ac80395b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:04:59.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.514+0000 7f5ad8ff9640 1 -- 172.21.15.60:0/2168297516 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f5acc00d4d0 con 0x7f5adc09cb20 2023-08-24T10:04:59.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:04:59.689+0000 7f5ae2663640 1 -- 172.21.15.60:0/2168297516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f5adc061800 con 0x7f5adc09cb20 2023-08-24T10:05:00.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.348+0000 7f5ad8ff9640 1 -- 172.21.15.60:0/2168297516 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f5acc00c6d0 con 0x7f5adc09cb20 2023-08-24T10:05:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.621+0000 7f5ad8ff9640 1 -- 172.21.15.60:0/2168297516 <== mon.0 v2:172.21.15.60:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v13) v1 ==== 109+0+0 (crc 0 0 0) 0x7f5acc00eb90 con 0x7f5adc09cb20 2023-08-24T10:05:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'rbd' created 2023-08-24T10:05:00.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.623+0000 7f5ae2663640 1 -- 172.21.15.60:0/2168297516 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5ac80370a0 msgr2=0x7f5ac80395b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:00.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.623+0000 7f5ae2663640 1 --2- 172.21.15.60:0/2168297516 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5ac80370a0 0x7f5ac80395b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:00.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.623+0000 7f5ae2663640 1 -- 172.21.15.60:0/2168297516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc09cb20 msgr2=0x7f5adc11bf10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:00.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.623+0000 7f5ae2663640 1 --2- 172.21.15.60:0/2168297516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc09cb20 0x7f5adc11bf10 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:00.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.624+0000 7f5ae2663640 1 -- 172.21.15.60:0/2168297516 shutdown_connections 2023-08-24T10:05:00.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.624+0000 7f5ae2663640 1 --2- 172.21.15.60:0/2168297516 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5ac80370a0 0x7f5ac80395b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:00.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.624+0000 7f5ae2663640 1 --2- 172.21.15.60:0/2168297516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc09cb20 0x7f5adc11bf10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:00.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.624+0000 7f5ae2663640 1 -- 172.21.15.60:0/2168297516 >> 172.21.15.60:0/2168297516 conn(0x7f5adc0a28a0 msgr2=0x7f5adc097a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:05:00.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.624+0000 7f5ae2663640 1 -- 172.21.15.60:0/2168297516 shutdown_connections 2023-08-24T10:05:00.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.624+0000 7f5ae2663640 1 -- 172.21.15.60:0/2168297516 wait complete. 2023-08-24T10:05:00.641 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-08-24T10:05:00.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.756+0000 7fa09b06c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:05:00.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.765+0000 7fa09b06c640 1 Processor -- start 2023-08-24T10:05:00.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.765+0000 7fa09b06c640 1 -- start start 2023-08-24T10:05:00.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.766+0000 7fa09b06c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa094132ae0 0x7fa094132ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:00.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.766+0000 7fa09b06c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0941325a0 con 0x7fa09412f0a0 2023-08-24T10:05:00.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.766+0000 7fa09b06c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa09409d950 con 0x7fa094132ae0 2023-08-24T10:05:00.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.766+0000 7fa098e08640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa094132ae0 0x7fa094132ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:00.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.766+0000 7fa093fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa09412f0a0 0x7fa09412f4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52216/0 (socket says 172.21.15.60:52216) 2023-08-24T10:05:00.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.766+0000 7fa093fff640 1 -- 172.21.15.60:0/3532027559 learned_addr learned my addr 172.21.15.60:0/3532027559 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:05:00.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.767+0000 7fa098e08640 1 -- 172.21.15.60:0/3532027559 >> v1:172.21.15.60:6789/0 conn(0x7fa09412f0a0 legacy=0x7fa09412f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:00.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.767+0000 7fa098e08640 1 -- 172.21.15.60:0/3532027559 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09409c1c0 con 0x7fa094132ae0 2023-08-24T10:05:00.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.767+0000 7fa098e08640 1 --2- 172.21.15.60:0/3532027559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa094132ae0 0x7fa094132ef0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8133c595d714edc9 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:00.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.767+0000 7fa0937fe640 1 -- 172.21.15.60:0/3532027559 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa088009ce0 con 0x7fa094132ae0 2023-08-24T10:05:00.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.767+0000 7fa0937fe640 1 -- 172.21.15.60:0/3532027559 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa088009e40 con 0x7fa094132ae0 2023-08-24T10:05:00.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.767+0000 7fa0937fe640 1 -- 172.21.15.60:0/3532027559 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa08800e360 con 0x7fa094132ae0 2023-08-24T10:05:00.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.768+0000 7fa09b06c640 1 -- 172.21.15.60:0/3532027559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa094132ae0 msgr2=0x7fa094132ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:00.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.768+0000 7fa09b06c640 1 --2- 172.21.15.60:0/3532027559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa094132ae0 0x7fa094132ef0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:00.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.768+0000 7fa09b06c640 1 -- 172.21.15.60:0/3532027559 shutdown_connections 2023-08-24T10:05:00.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.768+0000 7fa09b06c640 1 --2- 172.21.15.60:0/3532027559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa094132ae0 0x7fa094132ef0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:00.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.768+0000 7fa09b06c640 1 -- 172.21.15.60:0/3532027559 >> 172.21.15.60:0/3532027559 conn(0x7fa0940a2630 msgr2=0x7fa0940a2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:05:00.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.768+0000 7fa09b06c640 1 -- 172.21.15.60:0/3532027559 shutdown_connections 2023-08-24T10:05:00.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.768+0000 7fa09b06c640 1 -- 172.21.15.60:0/3532027559 wait complete. 2023-08-24T10:05:00.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.768+0000 7fa09b06c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:05:00.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.769+0000 7fa09b06c640 1 Processor -- start 2023-08-24T10:05:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.769+0000 7fa09b06c640 1 -- start start 2023-08-24T10:05:00.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.770+0000 7fa09b06c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa09412f0a0 0x7fa0940a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:00.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.770+0000 7fa09b06c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09409fc10 con 0x7fa09412f0a0 2023-08-24T10:05:00.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.770+0000 7fa098e08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa09412f0a0 0x7fa0940a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:00.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.770+0000 7fa098e08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa09412f0a0 0x7fa0940a00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33000/0 (socket says 172.21.15.60:33000) 2023-08-24T10:05:00.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.770+0000 7fa098e08640 1 -- 172.21.15.60:0/1739332125 learned_addr learned my addr 172.21.15.60:0/1739332125 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:05:00.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.771+0000 7fa098e08640 1 -- 172.21.15.60:0/1739332125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09409f5d0 con 0x7fa09412f0a0 2023-08-24T10:05:00.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.771+0000 7fa098e08640 1 --2- 172.21.15.60:0/1739332125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa09412f0a0 0x7fa0940a00d0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:00.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.771+0000 7fa0917fa640 1 -- 172.21.15.60:0/1739332125 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa088014540 con 0x7fa09412f0a0 2023-08-24T10:05:00.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.771+0000 7fa0917fa640 1 -- 172.21.15.60:0/1739332125 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa0880146a0 con 0x7fa09412f0a0 2023-08-24T10:05:00.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.771+0000 7fa09b06c640 1 -- 172.21.15.60:0/1739332125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09409f870 con 0x7fa09412f0a0 2023-08-24T10:05:00.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.771+0000 7fa0917fa640 1 -- 172.21.15.60:0/1739332125 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa088014940 con 0x7fa09412f0a0 2023-08-24T10:05:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.771+0000 7fa09b06c640 1 -- 172.21.15.60:0/1739332125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09411af20 con 0x7fa09412f0a0 2023-08-24T10:05:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.772+0000 7fa0917fa640 1 -- 172.21.15.60:0/1739332125 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fa088014b30 con 0x7fa09412f0a0 2023-08-24T10:05:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.772+0000 7fa0917fa640 1 --2- 172.21.15.60:0/1739332125 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa078037180 0x7fa078039690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.772+0000 7fa0917fa640 1 -- 172.21.15.60:0/1739332125 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2661+0+0 (crc 0 0 0) 0x7fa088019040 con 0x7fa09412f0a0 2023-08-24T10:05:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.772+0000 7fa09b06c640 1 -- 172.21.15.60:0/1739332125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa094061800 con 0x7fa09412f0a0 2023-08-24T10:05:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.772+0000 7fa093fff640 1 --2- 172.21.15.60:0/1739332125 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa078037180 0x7fa078039690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.773+0000 7fa093fff640 1 --2- 172.21.15.60:0/1739332125 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa078037180 0x7fa078039690 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:00.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.777+0000 7fa0917fa640 1 -- 172.21.15.60:0/1739332125 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fa088014e30 con 0x7fa09412f0a0 2023-08-24T10:05:00.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:00.956+0000 7fa09b06c640 1 -- 172.21.15.60:0/1739332125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa09409fa00 con 0x7fa09412f0a0 2023-08-24T10:05:01.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.622+0000 7fa0917fa640 1 -- 172.21.15.60:0/1739332125 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v14) v1 ==== 173+0+0 (crc 0 0 0) 0x7fa088005960 con 0x7fa09412f0a0 2023-08-24T10:05:01.623 INFO:teuthology.orchestra.run.smithi060.stderr:enabled application 'rbd' on pool 'rbd' 2023-08-24T10:05:01.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.625+0000 7fa09b06c640 1 -- 172.21.15.60:0/1739332125 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa078037180 msgr2=0x7fa078039690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:01.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.625+0000 7fa09b06c640 1 --2- 172.21.15.60:0/1739332125 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa078037180 0x7fa078039690 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:01.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.625+0000 7fa09b06c640 1 -- 172.21.15.60:0/1739332125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa09412f0a0 msgr2=0x7fa0940a00d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:01.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.625+0000 7fa09b06c640 1 --2- 172.21.15.60:0/1739332125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa09412f0a0 0x7fa0940a00d0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:01.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.625+0000 7fa09b06c640 1 -- 172.21.15.60:0/1739332125 shutdown_connections 2023-08-24T10:05:01.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.625+0000 7fa09b06c640 1 --2- 172.21.15.60:0/1739332125 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa078037180 0x7fa078039690 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:01.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.625+0000 7fa09b06c640 1 --2- 172.21.15.60:0/1739332125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa09412f0a0 0x7fa0940a00d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:01.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.625+0000 7fa09b06c640 1 -- 172.21.15.60:0/1739332125 >> 172.21.15.60:0/1739332125 conn(0x7fa0940a2630 msgr2=0x7fa094099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:05:01.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.625+0000 7fa09b06c640 1 -- 172.21.15.60:0/1739332125 shutdown_connections 2023-08-24T10:05:01.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.625+0000 7fa09b06c640 1 -- 172.21.15.60:0/1739332125 wait complete. 2023-08-24T10:05:01.642 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-08-24T10:05:01.643 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-08-24T10:05:01.643 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-08-24T10:05:01.643 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-08-24T10:05:01.644 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-08-24T10:05:01.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.755+0000 7f43b891e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:05:01.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.759+0000 7f43b891e640 1 Processor -- start 2023-08-24T10:05:01.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.759+0000 7f43b891e640 1 -- start start 2023-08-24T10:05:01.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.760+0000 7f43b891e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43b4133140 0x7f43b4137970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:01.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.760+0000 7f43b891e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b4133550 con 0x7f43b409c4a0 2023-08-24T10:05:01.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.760+0000 7f43b891e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b409d8d0 con 0x7f43b4133140 2023-08-24T10:05:01.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.760+0000 7f43b259c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f43b409c4a0 0x7f43b4132a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52224/0 (socket says 172.21.15.60:52224) 2023-08-24T10:05:01.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.760+0000 7f43b259c640 1 -- 172.21.15.60:0/3698677972 learned_addr learned my addr 172.21.15.60:0/3698677972 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:05:01.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.760+0000 7f43b1d9b640 1 --2- 172.21.15.60:0/3698677972 >> v2:172.21.15.60:3300/0 conn(0x7f43b4133140 0x7f43b4137970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:01.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.760+0000 7f43b159a640 1 -- 172.21.15.60:0/3698677972 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458043329 0 0) 0x7f43b4133550 con 0x7f43b409c4a0 2023-08-24T10:05:01.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.760+0000 7f43b159a640 1 -- 172.21.15.60:0/3698677972 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43a00034f0 con 0x7f43b409c4a0 2023-08-24T10:05:01.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.761+0000 7f43b1d9b640 1 -- 172.21.15.60:0/3698677972 >> v1:172.21.15.60:6789/0 conn(0x7f43b409c4a0 legacy=0x7f43b4132a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:01.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.761+0000 7f43b1d9b640 1 -- 172.21.15.60:0/3698677972 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b4132720 con 0x7f43b4133140 2023-08-24T10:05:01.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.761+0000 7f43b1d9b640 1 --2- 172.21.15.60:0/3698677972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b4133140 0x7f43b4137970 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41dcdb170df563ce server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:01.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.761+0000 7f43b159a640 1 -- 172.21.15.60:0/3698677972 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f439c009ce0 con 0x7f43b4133140 2023-08-24T10:05:01.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.761+0000 7f43b159a640 1 -- 172.21.15.60:0/3698677972 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f439c009e40 con 0x7f43b4133140 2023-08-24T10:05:01.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.761+0000 7f43b159a640 1 -- 172.21.15.60:0/3698677972 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f439c00e360 con 0x7f43b4133140 2023-08-24T10:05:01.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.762+0000 7f43b891e640 1 -- 172.21.15.60:0/3698677972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b4133140 msgr2=0x7f43b4137970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:01.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.762+0000 7f43b891e640 1 --2- 172.21.15.60:0/3698677972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b4133140 0x7f43b4137970 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:01.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.762+0000 7f43b891e640 1 -- 172.21.15.60:0/3698677972 shutdown_connections 2023-08-24T10:05:01.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.762+0000 7f43b891e640 1 --2- 172.21.15.60:0/3698677972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b4133140 0x7f43b4137970 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:01.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.762+0000 7f43b891e640 1 -- 172.21.15.60:0/3698677972 >> 172.21.15.60:0/3698677972 conn(0x7f43b40a2500 msgr2=0x7f43b40a2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:05:01.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.762+0000 7f43b891e640 1 -- 172.21.15.60:0/3698677972 shutdown_connections 2023-08-24T10:05:01.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.762+0000 7f43b891e640 1 -- 172.21.15.60:0/3698677972 wait complete. 2023-08-24T10:05:01.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.762+0000 7f43b891e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:05:01.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.763+0000 7f43b891e640 1 Processor -- start 2023-08-24T10:05:01.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.763+0000 7f43b891e640 1 -- start start 2023-08-24T10:05:01.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.764+0000 7f43b891e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b409c4a0 0x7f43b411c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:01.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.764+0000 7f43b891e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b411df40 con 0x7f43b409c4a0 2023-08-24T10:05:01.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.764+0000 7f43b259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b409c4a0 0x7f43b411c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:01.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.764+0000 7f43b259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b409c4a0 0x7f43b411c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33008/0 (socket says 172.21.15.60:33008) 2023-08-24T10:05:01.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.764+0000 7f43b259c640 1 -- 172.21.15.60:0/2566458037 learned_addr learned my addr 172.21.15.60:0/2566458037 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:05:01.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.765+0000 7f43b259c640 1 -- 172.21.15.60:0/2566458037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b411bf60 con 0x7f43b409c4a0 2023-08-24T10:05:01.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.765+0000 7f43b259c640 1 --2- 172.21.15.60:0/2566458037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b409c4a0 0x7f43b411c260 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:01.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.765+0000 7f43aaffd640 1 -- 172.21.15.60:0/2566458037 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f43a40059d0 con 0x7f43b409c4a0 2023-08-24T10:05:01.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.765+0000 7f43aaffd640 1 -- 172.21.15.60:0/2566458037 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f43a4005b30 con 0x7f43b409c4a0 2023-08-24T10:05:01.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.765+0000 7f43b891e640 1 -- 172.21.15.60:0/2566458037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b411bc50 con 0x7f43b409c4a0 2023-08-24T10:05:01.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.765+0000 7f43aaffd640 1 -- 172.21.15.60:0/2566458037 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f43a4005dd0 con 0x7f43b409c4a0 2023-08-24T10:05:01.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.765+0000 7f43b891e640 1 -- 172.21.15.60:0/2566458037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b411eb80 con 0x7f43b409c4a0 2023-08-24T10:05:01.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.766+0000 7f43b891e640 1 -- 172.21.15.60:0/2566458037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b411e080 con 0x7f43b409c4a0 2023-08-24T10:05:01.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.766+0000 7f43aaffd640 1 -- 172.21.15.60:0/2566458037 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f43a40070b0 con 0x7f43b409c4a0 2023-08-24T10:05:01.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.766+0000 7f43aaffd640 1 --2- 172.21.15.60:0/2566458037 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4394037130 0x7f4394039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:01.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.766+0000 7f43aaffd640 1 -- 172.21.15.60:0/2566458037 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f43a4046360 con 0x7f43b409c4a0 2023-08-24T10:05:01.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.767+0000 7f43b1d9b640 1 --2- 172.21.15.60:0/2566458037 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4394037130 0x7f4394039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:01.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.767+0000 7f43b1d9b640 1 --2- 172.21.15.60:0/2566458037 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4394037130 0x7f4394039640 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:01.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:01.775+0000 7f43aaffd640 1 -- 172.21.15.60:0/2566458037 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f43a400d330 con 0x7f43b409c4a0 2023-08-24T10:05:02.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.002+0000 7f43b891e640 1 -- 172.21.15.60:0/2566458037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f43b4067820 con 0x7f43b409c4a0 2023-08-24T10:05:02.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.003+0000 7f43aaffd640 1 -- 172.21.15.60:0/2566458037 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+85413 (crc 0 0 0) 0x7f43a400d150 con 0x7f43b409c4a0 2023-08-24T10:05:02.010 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:05:02.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.006+0000 7f43b891e640 1 -- 172.21.15.60:0/2566458037 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4394037130 msgr2=0x7f4394039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:02.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.006+0000 7f43b891e640 1 --2- 172.21.15.60:0/2566458037 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4394037130 0x7f4394039640 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.006+0000 7f43b891e640 1 -- 172.21.15.60:0/2566458037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b409c4a0 msgr2=0x7f43b411c260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:02.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.006+0000 7f43b891e640 1 --2- 172.21.15.60:0/2566458037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b409c4a0 0x7f43b411c260 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.006+0000 7f43b891e640 1 -- 172.21.15.60:0/2566458037 shutdown_connections 2023-08-24T10:05:02.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.006+0000 7f43b891e640 1 --2- 172.21.15.60:0/2566458037 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4394037130 0x7f4394039640 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.006+0000 7f43b891e640 1 --2- 172.21.15.60:0/2566458037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b409c4a0 0x7f43b411c260 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.006+0000 7f43b891e640 1 -- 172.21.15.60:0/2566458037 >> 172.21.15.60:0/2566458037 conn(0x7f43b40a2500 msgr2=0x7f43b4099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:05:02.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.006+0000 7f43b891e640 1 -- 172.21.15.60:0/2566458037 shutdown_connections 2023-08-24T10:05:02.013 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.006+0000 7f43b891e640 1 -- 172.21.15.60:0/2566458037 wait complete. 2023-08-24T10:05:02.017 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":5,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3444395587},{"type":"v1","addr":"172.21.15.135:6801","nonce":3444395587}]},"active_addr":"172.21.15.135:6801/3444395587","active_change":"2023-08-24T10:04:53.491993+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":2661590335}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":3945351250}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":2094155590}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":2000390485}]}]} 2023-08-24T10:05:02.020 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-08-24T10:05:02.020 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-08-24T10:05:02.021 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-24T10:05:02.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.127+0000 7f797326c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:05:02.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.131+0000 7f797326c640 1 Processor -- start 2023-08-24T10:05:02.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.131+0000 7f797326c640 1 -- start start 2023-08-24T10:05:02.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.132+0000 7f797326c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f796c06d300 0x7f796c12bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.132+0000 7f797326c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f796c099440 con 0x7f796c12c1c0 2023-08-24T10:05:02.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.132+0000 7f797326c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f796c09a050 con 0x7f796c06d300 2023-08-24T10:05:02.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.132+0000 7f7971008640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f796c06d300 0x7f796c12bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:02.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.132+0000 7f7971008640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f796c06d300 0x7f796c12bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33018/0 (socket says 172.21.15.60:33018) 2023-08-24T10:05:02.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.132+0000 7f7970807640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f796c12c1c0 0x7f796c12ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52238/0 (socket says 172.21.15.60:52238) 2023-08-24T10:05:02.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.132+0000 7f7971008640 1 -- 172.21.15.60:0/2987186092 learned_addr learned my addr 172.21.15.60:0/2987186092 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:05:02.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.132+0000 7f7971008640 1 -- 172.21.15.60:0/2987186092 >> v1:172.21.15.60:6789/0 conn(0x7f796c12c1c0 legacy=0x7f796c12ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:02.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.132+0000 7f7971008640 1 -- 172.21.15.60:0/2987186092 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c12b9d0 con 0x7f796c06d300 2023-08-24T10:05:02.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.133+0000 7f7971008640 1 --2- 172.21.15.60:0/2987186092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f796c06d300 0x7f796c12bc80 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69bbdcfdb29cb29 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:02.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.133+0000 7f7963fff640 1 -- 172.21.15.60:0/2987186092 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f796800ad60 con 0x7f796c06d300 2023-08-24T10:05:02.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.133+0000 7f7963fff640 1 -- 172.21.15.60:0/2987186092 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f796800f340 con 0x7f796c06d300 2023-08-24T10:05:02.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.133+0000 7f7963fff640 1 -- 172.21.15.60:0/2987186092 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f796800f5e0 con 0x7f796c06d300 2023-08-24T10:05:02.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.133+0000 7f797326c640 1 -- 172.21.15.60:0/2987186092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f796c06d300 msgr2=0x7f796c12bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:02.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.133+0000 7f797326c640 1 --2- 172.21.15.60:0/2987186092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f796c06d300 0x7f796c12bc80 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.133+0000 7f797326c640 1 -- 172.21.15.60:0/2987186092 shutdown_connections 2023-08-24T10:05:02.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.133+0000 7f797326c640 1 --2- 172.21.15.60:0/2987186092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f796c06d300 0x7f796c12bc80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.133+0000 7f797326c640 1 -- 172.21.15.60:0/2987186092 >> 172.21.15.60:0/2987186092 conn(0x7f796c09e110 msgr2=0x7f796c09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:05:02.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.133+0000 7f797326c640 1 -- 172.21.15.60:0/2987186092 shutdown_connections 2023-08-24T10:05:02.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.134+0000 7f797326c640 1 -- 172.21.15.60:0/2987186092 wait complete. 2023-08-24T10:05:02.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.134+0000 7f797326c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:05:02.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.135+0000 7f797326c640 1 Processor -- start 2023-08-24T10:05:02.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.135+0000 7f797326c640 1 -- start start 2023-08-24T10:05:02.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.135+0000 7f797326c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f796c06d300 0x7f796c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.135+0000 7f797326c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c089370 con 0x7f796c06d300 2023-08-24T10:05:02.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.135+0000 7f7971008640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f796c06d300 0x7f796c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:02.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.135+0000 7f7971008640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f796c06d300 0x7f796c06e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33032/0 (socket says 172.21.15.60:33032) 2023-08-24T10:05:02.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.135+0000 7f7971008640 1 -- 172.21.15.60:0/2604779370 learned_addr learned my addr 172.21.15.60:0/2604779370 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:05:02.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.136+0000 7f7971008640 1 -- 172.21.15.60:0/2604779370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c096f10 con 0x7f796c06d300 2023-08-24T10:05:02.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.136+0000 7f7971008640 1 --2- 172.21.15.60:0/2604779370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f796c06d300 0x7f796c06e9f0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:02.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.136+0000 7f7961ffb640 1 -- 172.21.15.60:0/2604779370 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7968017040 con 0x7f796c06d300 2023-08-24T10:05:02.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.136+0000 7f7961ffb640 1 -- 172.21.15.60:0/2604779370 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f796800fdf0 con 0x7f796c06d300 2023-08-24T10:05:02.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.136+0000 7f797326c640 1 -- 172.21.15.60:0/2604779370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796c089560 con 0x7f796c06d300 2023-08-24T10:05:02.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.136+0000 7f7961ffb640 1 -- 172.21.15.60:0/2604779370 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7968011550 con 0x7f796c06d300 2023-08-24T10:05:02.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.136+0000 7f797326c640 1 -- 172.21.15.60:0/2604779370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796c088ff0 con 0x7f796c06d300 2023-08-24T10:05:02.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.137+0000 7f7961ffb640 1 -- 172.21.15.60:0/2604779370 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f7968011700 con 0x7f796c06d300 2023-08-24T10:05:02.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.137+0000 7f7961ffb640 1 --2- 172.21.15.60:0/2604779370 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7950037130 0x7f7950039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.137+0000 7f7961ffb640 1 -- 172.21.15.60:0/2604779370 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f79680484d0 con 0x7f796c06d300 2023-08-24T10:05:02.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.137+0000 7f7970807640 1 --2- 172.21.15.60:0/2604779370 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7950037130 0x7f7950039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:02.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.137+0000 7f797326c640 1 -- 172.21.15.60:0/2604779370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f796c06e840 con 0x7f796c06d300 2023-08-24T10:05:02.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.138+0000 7f7970807640 1 --2- 172.21.15.60:0/2604779370 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7950037130 0x7f7950039640 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:02.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.142+0000 7f7961ffb640 1 -- 172.21.15.60:0/2604779370 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f79680103b0 con 0x7f796c06d300 2023-08-24T10:05:02.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.319+0000 7f797326c640 1 -- 172.21.15.60:0/2604779370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f796c061800 con 0x7f796c06d300 2023-08-24T10:05:02.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.320+0000 7f7961ffb640 1 -- 172.21.15.60:0/2604779370 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+8671 (crc 0 0 0) 0x7f796800c250 con 0x7f796c06d300 2023-08-24T10:05:02.321 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:05:02.321 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":14,"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","created":"2023-08-24T10:04:49.373034+0000","modified":"2023-08-24T10:05:01.619965+0000","last_up_change":"2023-08-24T10:04:55.538474+0000","last_in_change":"2023-08-24T10:04:51.502866+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-24T10:04:57.553657+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-24T10:04:59.720212+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72632e8e-7a21-476f-91d4-c34b43117b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2841298624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":2841298624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2841298624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":2841298624}]},"public_addr":"172.21.15.60:6800/2841298624","cluster_addr":"172.21.15.60:6801/2841298624","heartbeat_back_addr":"172.21.15.60:6804/2841298624","heartbeat_front_addr":"172.21.15.60:6805/2841298624","state":["exists","up"]},{"osd":1,"uuid":"3157bd05-35e6-4d2c-9e8b-ec16937c5e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2103781563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6803","nonce":2103781563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6807","nonce":2103781563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2103781563}]},"public_addr":"172.21.15.60:6802/2103781563","cluster_addr":"172.21.15.60:6803/2103781563","heartbeat_back_addr":"172.21.15.60:6807/2103781563","heartbeat_front_addr":"172.21.15.60:6806/2103781563","state":["exists","up"]},{"osd":2,"uuid":"77ec2ee0-440e-41ae-ab0b-0427531dfa8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1298459676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6805","nonce":1298459676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1298459676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":1298459676}]},"public_addr":"172.21.15.135:6804/1298459676","cluster_addr":"172.21.15.135:6805/1298459676","heartbeat_back_addr":"172.21.15.135:6808/1298459676","heartbeat_front_addr":"172.21.15.135:6809/1298459676","state":["exists","up"]},{"osd":3,"uuid":"9a042bd6-2e19-486f-a3df-91d939324042","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3144688060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6803","nonce":3144688060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6807","nonce":3144688060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3144688060}]},"public_addr":"172.21.15.135:6802/3144688060","cluster_addr":"172.21.15.135:6803/3144688060","heartbeat_back_addr":"172.21.15.135:6807/3144688060","heartbeat_front_addr":"172.21.15.135:6806/3144688060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-24T10:05:02.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.321+0000 7f797326c640 1 -- 172.21.15.60:0/2604779370 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7950037130 msgr2=0x7f7950039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:02.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.321+0000 7f797326c640 1 --2- 172.21.15.60:0/2604779370 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7950037130 0x7f7950039640 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.321+0000 7f797326c640 1 -- 172.21.15.60:0/2604779370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f796c06d300 msgr2=0x7f796c06e9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:02.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.322+0000 7f797326c640 1 --2- 172.21.15.60:0/2604779370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f796c06d300 0x7f796c06e9f0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.322+0000 7f797326c640 1 -- 172.21.15.60:0/2604779370 shutdown_connections 2023-08-24T10:05:02.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.322+0000 7f797326c640 1 --2- 172.21.15.60:0/2604779370 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7950037130 0x7f7950039640 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.322+0000 7f797326c640 1 --2- 172.21.15.60:0/2604779370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f796c06d300 0x7f796c06e9f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.322+0000 7f797326c640 1 -- 172.21.15.60:0/2604779370 >> 172.21.15.60:0/2604779370 conn(0x7f796c09e110 msgr2=0x7f796c09bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:05:02.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.322+0000 7f797326c640 1 -- 172.21.15.60:0/2604779370 shutdown_connections 2023-08-24T10:05:02.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.322+0000 7f797326c640 1 -- 172.21.15.60:0/2604779370 wait complete. 2023-08-24T10:05:02.334 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-08-24T10:05:02.334 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-24T10:05:02.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.442+0000 7f14fd26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:05:02.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.446+0000 7f14fd26c640 1 Processor -- start 2023-08-24T10:05:02.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.447+0000 7f14fd26c640 1 -- start start 2023-08-24T10:05:02.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.447+0000 7f14fd26c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f14f8132630 0x7f14f8134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.447+0000 7f14fd26c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14f806e320 con 0x7f14f8096860 2023-08-24T10:05:02.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.447+0000 7f14fd26c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f806db70 con 0x7f14f8132630 2023-08-24T10:05:02.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.447+0000 7f14f67fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f14f8132630 0x7f14f8134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:02.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.447+0000 7f14f67fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f14f8132630 0x7f14f8134a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33040/0 (socket says 172.21.15.60:33040) 2023-08-24T10:05:02.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.447+0000 7f14f67fc640 1 -- 172.21.15.60:0/4080712481 learned_addr learned my addr 172.21.15.60:0/4080712481 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:05:02.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.448+0000 7f14f67fc640 1 -- 172.21.15.60:0/4080712481 >> v1:172.21.15.60:6789/0 conn(0x7f14f8096860 legacy=0x7f14f8096c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:02.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.448+0000 7f14f67fc640 1 -- 172.21.15.60:0/4080712481 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f806cef0 con 0x7f14f8132630 2023-08-24T10:05:02.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.448+0000 7f14f67fc640 1 --2- 172.21.15.60:0/4080712481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8132630 0x7f14f8134a50 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6803aadbf1a76a1 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:02.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.448+0000 7f14f5ffb640 1 -- 172.21.15.60:0/4080712481 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f14e0009dd0 con 0x7f14f8132630 2023-08-24T10:05:02.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.448+0000 7f14f5ffb640 1 -- 172.21.15.60:0/4080712481 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f14e000e230 con 0x7f14f8132630 2023-08-24T10:05:02.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.448+0000 7f14f5ffb640 1 -- 172.21.15.60:0/4080712481 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f14e000e4d0 con 0x7f14f8132630 2023-08-24T10:05:02.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.448+0000 7f14fd26c640 1 -- 172.21.15.60:0/4080712481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8132630 msgr2=0x7f14f8134a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:02.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.448+0000 7f14fd26c640 1 --2- 172.21.15.60:0/4080712481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8132630 0x7f14f8134a50 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.449+0000 7f14fd26c640 1 -- 172.21.15.60:0/4080712481 shutdown_connections 2023-08-24T10:05:02.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.449+0000 7f14fd26c640 1 --2- 172.21.15.60:0/4080712481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8132630 0x7f14f8134a50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.449+0000 7f14fd26c640 1 -- 172.21.15.60:0/4080712481 >> 172.21.15.60:0/4080712481 conn(0x7f14f809e110 msgr2=0x7f14f809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:05:02.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.449+0000 7f14fd26c640 1 -- 172.21.15.60:0/4080712481 shutdown_connections 2023-08-24T10:05:02.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.449+0000 7f14fd26c640 1 -- 172.21.15.60:0/4080712481 wait complete. 2023-08-24T10:05:02.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.449+0000 7f14fd26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:05:02.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.450+0000 7f14fd26c640 1 Processor -- start 2023-08-24T10:05:02.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.450+0000 7f14fd26c640 1 -- start start 2023-08-24T10:05:02.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.451+0000 7f14fd26c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8096860 0x7f14f812f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.451+0000 7f14fd26c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f8128df0 con 0x7f14f8096860 2023-08-24T10:05:02.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.451+0000 7f14f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8096860 0x7f14f812f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:02.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.451+0000 7f14f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8096860 0x7f14f812f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33052/0 (socket says 172.21.15.60:33052) 2023-08-24T10:05:02.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.451+0000 7f14f6ffd640 1 -- 172.21.15.60:0/3655148498 learned_addr learned my addr 172.21.15.60:0/3655148498 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:05:02.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.451+0000 7f14f6ffd640 1 -- 172.21.15.60:0/3655148498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f81293a0 con 0x7f14f8096860 2023-08-24T10:05:02.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.451+0000 7f14f6ffd640 1 --2- 172.21.15.60:0/3655148498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8096860 0x7f14f812f690 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:02.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.451+0000 7f14ef7fe640 1 -- 172.21.15.60:0/3655148498 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f14e800cb60 con 0x7f14f8096860 2023-08-24T10:05:02.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.451+0000 7f14ef7fe640 1 -- 172.21.15.60:0/3655148498 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f14e800ccc0 con 0x7f14f8096860 2023-08-24T10:05:02.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.451+0000 7f14fd26c640 1 -- 172.21.15.60:0/3655148498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f8129090 con 0x7f14f8096860 2023-08-24T10:05:02.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.451+0000 7f14ef7fe640 1 -- 172.21.15.60:0/3655148498 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f14e800f230 con 0x7f14f8096860 2023-08-24T10:05:02.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.451+0000 7f14fd26c640 1 -- 172.21.15.60:0/3655148498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f8128720 con 0x7f14f8096860 2023-08-24T10:05:02.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.452+0000 7f14ef7fe640 1 -- 172.21.15.60:0/3655148498 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f14e800f3e0 con 0x7f14f8096860 2023-08-24T10:05:02.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.452+0000 7f14fd26c640 1 -- 172.21.15.60:0/3655148498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f8128c20 con 0x7f14f8096860 2023-08-24T10:05:02.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.452+0000 7f14ef7fe640 1 --2- 172.21.15.60:0/3655148498 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f14d8037130 0x7f14d8039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.452+0000 7f14ef7fe640 1 -- 172.21.15.60:0/3655148498 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f14e8046e90 con 0x7f14f8096860 2023-08-24T10:05:02.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.453+0000 7f14f67fc640 1 --2- 172.21.15.60:0/3655148498 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f14d8037130 0x7f14d8039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:02.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.453+0000 7f14f67fc640 1 --2- 172.21.15.60:0/3655148498 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f14d8037130 0x7f14d8039640 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:02.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.457+0000 7f14ef7fe640 1 -- 172.21.15.60:0/3655148498 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f14e800aab0 con 0x7f14f8096860 2023-08-24T10:05:02.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.635+0000 7f14fd26c640 1 -- 172.21.15.60:0/3655148498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14f8061800 con 0x7f14f8096860 2023-08-24T10:05:02.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.636+0000 7f14ef7fe640 1 -- 172.21.15.60:0/3655148498 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+8671 (crc 0 0 0) 0x7f14e800a8d0 con 0x7f14f8096860 2023-08-24T10:05:02.637 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:05:02.638 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":14,"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","created":"2023-08-24T10:04:49.373034+0000","modified":"2023-08-24T10:05:01.619965+0000","last_up_change":"2023-08-24T10:04:55.538474+0000","last_in_change":"2023-08-24T10:04:51.502866+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-24T10:04:57.553657+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-24T10:04:59.720212+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72632e8e-7a21-476f-91d4-c34b43117b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2841298624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6801","nonce":2841298624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2841298624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6805","nonce":2841298624}]},"public_addr":"172.21.15.60:6800/2841298624","cluster_addr":"172.21.15.60:6801/2841298624","heartbeat_back_addr":"172.21.15.60:6804/2841298624","heartbeat_front_addr":"172.21.15.60:6805/2841298624","state":["exists","up"]},{"osd":1,"uuid":"3157bd05-35e6-4d2c-9e8b-ec16937c5e5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2103781563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6803","nonce":2103781563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6807","nonce":2103781563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2103781563}]},"public_addr":"172.21.15.60:6802/2103781563","cluster_addr":"172.21.15.60:6803/2103781563","heartbeat_back_addr":"172.21.15.60:6807/2103781563","heartbeat_front_addr":"172.21.15.60:6806/2103781563","state":["exists","up"]},{"osd":2,"uuid":"77ec2ee0-440e-41ae-ab0b-0427531dfa8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1298459676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6805","nonce":1298459676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":1298459676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6809","nonce":1298459676}]},"public_addr":"172.21.15.135:6804/1298459676","cluster_addr":"172.21.15.135:6805/1298459676","heartbeat_back_addr":"172.21.15.135:6808/1298459676","heartbeat_front_addr":"172.21.15.135:6809/1298459676","state":["exists","up"]},{"osd":3,"uuid":"9a042bd6-2e19-486f-a3df-91d939324042","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3144688060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6803","nonce":3144688060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6807","nonce":3144688060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3144688060}]},"public_addr":"172.21.15.135:6802/3144688060","cluster_addr":"172.21.15.135:6803/3144688060","heartbeat_back_addr":"172.21.15.135:6807/3144688060","heartbeat_front_addr":"172.21.15.135:6806/3144688060","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-24T10:05:02.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.638+0000 7f14fd26c640 1 -- 172.21.15.60:0/3655148498 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f14d8037130 msgr2=0x7f14d8039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:02.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.638+0000 7f14fd26c640 1 --2- 172.21.15.60:0/3655148498 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f14d8037130 0x7f14d8039640 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.638+0000 7f14fd26c640 1 -- 172.21.15.60:0/3655148498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8096860 msgr2=0x7f14f812f690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:02.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.638+0000 7f14fd26c640 1 --2- 172.21.15.60:0/3655148498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8096860 0x7f14f812f690 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.638+0000 7f14fd26c640 1 -- 172.21.15.60:0/3655148498 shutdown_connections 2023-08-24T10:05:02.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.638+0000 7f14fd26c640 1 --2- 172.21.15.60:0/3655148498 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f14d8037130 0x7f14d8039640 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.638+0000 7f14fd26c640 1 --2- 172.21.15.60:0/3655148498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14f8096860 0x7f14f812f690 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.638+0000 7f14fd26c640 1 -- 172.21.15.60:0/3655148498 >> 172.21.15.60:0/3655148498 conn(0x7f14f809e110 msgr2=0x7f14f809b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:05:02.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.638+0000 7f14fd26c640 1 -- 172.21.15.60:0/3655148498 shutdown_connections 2023-08-24T10:05:02.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.638+0000 7f14fd26c640 1 -- 172.21.15.60:0/3655148498 wait complete. 2023-08-24T10:05:02.650 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-08-24T10:05:02.651 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-08-24T10:05:02.652 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-08-24T10:05:02.652 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-08-24T10:05:02.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.769+0000 7f8afdaec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:05:02.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.769+0000 7f500feec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:05:02.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.769+0000 7fc50f19e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:05:02.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.769+0000 7f148986c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:05:02.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.774+0000 7f500feec640 1 Processor -- start 2023-08-24T10:05:02.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.774+0000 7f148986c640 1 Processor -- start 2023-08-24T10:05:02.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.774+0000 7f8afdaec640 1 Processor -- start 2023-08-24T10:05:02.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.774+0000 7f500feec640 1 -- start start 2023-08-24T10:05:02.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.774+0000 7fc50f19e640 1 Processor -- start 2023-08-24T10:05:02.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.774+0000 7f148986c640 1 -- start start 2023-08-24T10:05:02.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.774+0000 7f8afdaec640 1 -- start start 2023-08-24T10:05:02.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.774+0000 7fc50f19e640 1 -- start start 2023-08-24T10:05:02.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7f500feec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f500806d300 0x7f500812bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7f148986c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f148406f3c0 0x7f148406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7f8afdaec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8af8132b50 0x7f8af8132f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7f500feec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5008099440 con 0x7f500812c1c0 2023-08-24T10:05:02.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7f500feec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f500809a050 con 0x7f500806d300 2023-08-24T10:05:02.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7fc50f19e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc50806fa30 0x7fc508098cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7fc50f19e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5080709e0 con 0x7fc50812d040 2023-08-24T10:05:02.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7fc50f19e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc508099230 con 0x7fc50806fa30 2023-08-24T10:05:02.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7f148986c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1484099830 con 0x7f148406fd10 2023-08-24T10:05:02.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7f148986c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1484099a60 con 0x7f148406f3c0 2023-08-24T10:05:02.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7f8afdaec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af8132610 con 0x7f8af812f110 2023-08-24T10:05:02.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7f8afdaec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af809c260 con 0x7f8af8132b50 2023-08-24T10:05:02.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7f8af77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8af8132b50 0x7f8af8132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:02.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7fc50cf3a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc50806fa30 0x7fc508098cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:02.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7f8af77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8af8132b50 0x7f8af8132f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33078/0 (socket says 172.21.15.60:33078) 2023-08-24T10:05:02.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7f8af77fe640 1 -- 172.21.15.60:0/2182402875 learned_addr learned my addr 172.21.15.60:0/2182402875 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:05:02.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7fc50cf3a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc50806fa30 0x7fc508098cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33084/0 (socket says 172.21.15.60:33084) 2023-08-24T10:05:02.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7fc50cf3a640 1 -- 172.21.15.60:0/1463851598 learned_addr learned my addr 172.21.15.60:0/1463851598 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:05:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7f500dc88640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f500806d300 0x7f500812bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7f500dc88640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f500806d300 0x7f500812bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33058/0 (socket says 172.21.15.60:33058) 2023-08-24T10:05:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7f500dc88640 1 -- 172.21.15.60:0/1050467801 learned_addr learned my addr 172.21.15.60:0/1050467801 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:05:02.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7f14827fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f148406fd10 0x7f1484130530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52254/0 (socket says 172.21.15.60:52254) 2023-08-24T10:05:02.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7f14827fc640 1 -- 172.21.15.60:0/784205851 learned_addr learned my addr 172.21.15.60:0/784205851 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:05:02.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.775+0000 7f1482ffd640 1 --2- 172.21.15.60:0/784205851 >> v2:172.21.15.60:3300/0 conn(0x7f148406f3c0 0x7f148406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:02.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7fc50cf3a640 1 -- 172.21.15.60:0/1463851598 >> v1:172.21.15.60:6789/0 conn(0x7fc50812d040 legacy=0x7fc508130530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:02.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7fc50cf3a640 1 -- 172.21.15.60:0/1463851598 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50812d8d0 con 0x7fc50806fa30 2023-08-24T10:05:02.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7f8af77fe640 1 -- 172.21.15.60:0/2182402875 >> v1:172.21.15.60:6789/0 conn(0x7f8af812f110 legacy=0x7f8af812f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-24T10:05:02.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7f8af77fe640 1 -- 172.21.15.60:0/2182402875 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af8132840 con 0x7f8af8132b50 2023-08-24T10:05:02.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7f1481ffb640 1 -- 172.21.15.60:0/784205851 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3109135522 0 0) 0x7f1484099830 con 0x7f148406fd10 2023-08-24T10:05:02.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7f1481ffb640 1 -- 172.21.15.60:0/784205851 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f146c0034f0 con 0x7f148406fd10 2023-08-24T10:05:02.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7f500dc88640 1 -- 172.21.15.60:0/1050467801 >> v1:172.21.15.60:6789/0 conn(0x7f500812c1c0 legacy=0x7f500812ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:02.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7fc50cf3a640 1 --2- 172.21.15.60:0/1463851598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc50806fa30 0x7fc508098cf0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3377fd3148df6a server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:02.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7f500dc88640 1 -- 172.21.15.60:0/1050467801 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500812b9d0 con 0x7f500806d300 2023-08-24T10:05:02.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7f1482ffd640 1 -- 172.21.15.60:0/784205851 >> v1:172.21.15.60:6789/0 conn(0x7f148406fd10 legacy=0x7f1484130530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:02.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7f1482ffd640 1 -- 172.21.15.60:0/784205851 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148409c7e0 con 0x7f148406f3c0 2023-08-24T10:05:02.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7f8af77fe640 1 --2- 172.21.15.60:0/2182402875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8af8132b50 0x7f8af8132f60 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4060714075d5c521 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:02.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7fc5077fe640 1 -- 172.21.15.60:0/1463851598 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc4fc009dd0 con 0x7fc50806fa30 2023-08-24T10:05:02.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7fc5077fe640 1 -- 172.21.15.60:0/1463851598 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc4fc00e230 con 0x7fc50806fa30 2023-08-24T10:05:02.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7fc5077fe640 1 -- 172.21.15.60:0/1463851598 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc4fc00e4d0 con 0x7fc50806fa30 2023-08-24T10:05:02.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7f8af67fc640 1 -- 172.21.15.60:0/2182402875 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8ae8009dd0 con 0x7f8af8132b50 2023-08-24T10:05:02.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7f8af67fc640 1 -- 172.21.15.60:0/2182402875 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f8ae800e230 con 0x7f8af8132b50 2023-08-24T10:05:02.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7f8af67fc640 1 -- 172.21.15.60:0/2182402875 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8ae800e4d0 con 0x7f8af8132b50 2023-08-24T10:05:02.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7f500dc88640 1 --2- 172.21.15.60:0/1050467801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f500806d300 0x7f500812bc80 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed0d29804247f73 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:02.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.776+0000 7f1482ffd640 1 --2- 172.21.15.60:0/784205851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f148406f3c0 0x7f148406f7d0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9433a82346eaac server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:02.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.777+0000 7f500cc86640 1 -- 172.21.15.60:0/1050467801 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f500400ad60 con 0x7f500806d300 2023-08-24T10:05:02.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.777+0000 7f500cc86640 1 -- 172.21.15.60:0/1050467801 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f500400f340 con 0x7f500806d300 2023-08-24T10:05:02.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.777+0000 7f500cc86640 1 -- 172.21.15.60:0/1050467801 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f500400f5e0 con 0x7f500806d300 2023-08-24T10:05:02.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.777+0000 7f1481ffb640 1 -- 172.21.15.60:0/784205851 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1474009ce0 con 0x7f148406f3c0 2023-08-24T10:05:02.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.777+0000 7f1481ffb640 1 -- 172.21.15.60:0/784205851 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1474009e40 con 0x7f148406f3c0 2023-08-24T10:05:02.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.777+0000 7f1481ffb640 1 -- 172.21.15.60:0/784205851 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f147400e360 con 0x7f148406f3c0 2023-08-24T10:05:02.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.777+0000 7f8afdaec640 1 -- 172.21.15.60:0/2182402875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8af8132b50 msgr2=0x7f8af8132f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:02.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.777+0000 7f8afdaec640 1 --2- 172.21.15.60:0/2182402875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8af8132b50 0x7f8af8132f60 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.777+0000 7fc50f19e640 1 -- 172.21.15.60:0/1463851598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc50806fa30 msgr2=0x7fc508098cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:02.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.777+0000 7fc50f19e640 1 --2- 172.21.15.60:0/1463851598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc50806fa30 0x7fc508098cf0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.777+0000 7f8afdaec640 1 -- 172.21.15.60:0/2182402875 shutdown_connections 2023-08-24T10:05:02.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f8afdaec640 1 --2- 172.21.15.60:0/2182402875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8af8132b50 0x7f8af8132f60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f8afdaec640 1 -- 172.21.15.60:0/2182402875 >> 172.21.15.60:0/2182402875 conn(0x7f8af80a26a0 msgr2=0x7f8af80a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:05:02.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f500feec640 1 -- 172.21.15.60:0/1050467801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f500806d300 msgr2=0x7f500812bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:02.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f500feec640 1 --2- 172.21.15.60:0/1050467801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f500806d300 0x7f500812bc80 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7fc50f19e640 1 -- 172.21.15.60:0/1463851598 shutdown_connections 2023-08-24T10:05:02.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7fc50f19e640 1 --2- 172.21.15.60:0/1463851598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc50806fa30 0x7fc508098cf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7fc50f19e640 1 -- 172.21.15.60:0/1463851598 >> 172.21.15.60:0/1463851598 conn(0x7fc5080a2700 msgr2=0x7fc5080a2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:05:02.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f8afdaec640 1 -- 172.21.15.60:0/2182402875 shutdown_connections 2023-08-24T10:05:02.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f500feec640 1 -- 172.21.15.60:0/1050467801 shutdown_connections 2023-08-24T10:05:02.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f500feec640 1 --2- 172.21.15.60:0/1050467801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f500806d300 0x7f500812bc80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7fc50f19e640 1 -- 172.21.15.60:0/1463851598 shutdown_connections 2023-08-24T10:05:02.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f500feec640 1 -- 172.21.15.60:0/1050467801 >> 172.21.15.60:0/1050467801 conn(0x7f500809e110 msgr2=0x7f500809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:05:02.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7fc50f19e640 1 -- 172.21.15.60:0/1463851598 wait complete. 2023-08-24T10:05:02.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f8afdaec640 1 -- 172.21.15.60:0/2182402875 wait complete. 2023-08-24T10:05:02.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f500feec640 1 -- 172.21.15.60:0/1050467801 shutdown_connections 2023-08-24T10:05:02.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f500feec640 1 -- 172.21.15.60:0/1050467801 wait complete. 2023-08-24T10:05:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f148986c640 1 -- 172.21.15.60:0/784205851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f148406f3c0 msgr2=0x7f148406f7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:05:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f148986c640 1 --2- 172.21.15.60:0/784205851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f148406f3c0 0x7f148406f7d0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7fc50f19e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:05:02.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f8afdaec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:05:02.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f148986c640 1 -- 172.21.15.60:0/784205851 shutdown_connections 2023-08-24T10:05:02.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f148986c640 1 --2- 172.21.15.60:0/784205851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f148406f3c0 0x7f148406f7d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:05:02.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f148986c640 1 -- 172.21.15.60:0/784205851 >> 172.21.15.60:0/784205851 conn(0x7f14840a24d0 msgr2=0x7f14840a28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:05:02.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f148986c640 1 -- 172.21.15.60:0/784205851 shutdown_connections 2023-08-24T10:05:02.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f148986c640 1 -- 172.21.15.60:0/784205851 wait complete. 2023-08-24T10:05:02.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f500feec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:05:02.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.778+0000 7f148986c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:05:02.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.779+0000 7f8afdaec640 1 Processor -- start 2023-08-24T10:05:02.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.779+0000 7fc50f19e640 1 Processor -- start 2023-08-24T10:05:02.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.779+0000 7fc50f19e640 1 -- start start 2023-08-24T10:05:02.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.779+0000 7f8afdaec640 1 -- start start 2023-08-24T10:05:02.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.780+0000 7f500feec640 1 Processor -- start 2023-08-24T10:05:02.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.780+0000 7f500feec640 1 -- start start 2023-08-24T10:05:02.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.780+0000 7f148986c640 1 Processor -- start 2023-08-24T10:05:02.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.780+0000 7f8afdaec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8af812f110 0x7f8af809fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.780+0000 7f8afdaec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af812cc80 con 0x7f8af812f110 2023-08-24T10:05:02.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.780+0000 7fc50f19e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc50806fa30 0x7fc508129a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.780+0000 7fc50f19e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50812ede0 con 0x7fc50806fa30 2023-08-24T10:05:02.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.780+0000 7f8af77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8af812f110 0x7f8af809fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:02.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.780+0000 7f8af77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8af812f110 0x7f8af809fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33100/0 (socket says 172.21.15.60:33100) 2023-08-24T10:05:02.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.780+0000 7f8af77fe640 1 -- 172.21.15.60:0/939571867 learned_addr learned my addr 172.21.15.60:0/939571867 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:05:02.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.780+0000 7f148986c640 1 -- start start 2023-08-24T10:05:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.780+0000 7fc50cf3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc50806fa30 0x7fc508129a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.780+0000 7fc50cf3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc50806fa30 0x7fc508129a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33102/0 (socket says 172.21.15.60:33102) 2023-08-24T10:05:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.780+0000 7fc50cf3a640 1 -- 172.21.15.60:0/835603021 learned_addr learned my addr 172.21.15.60:0/835603021 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:05:02.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.780+0000 7f8af77fe640 1 -- 172.21.15.60:0/939571867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af809f400 con 0x7f8af812f110 2023-08-24T10:05:02.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.781+0000 7fc50cf3a640 1 -- 172.21.15.60:0/835603021 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc508130690 con 0x7fc50806fa30 2023-08-24T10:05:02.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.781+0000 7f8af77fe640 1 --2- 172.21.15.60:0/939571867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8af812f110 0x7f8af809fcd0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:02.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.781+0000 7fc50cf3a640 1 --2- 172.21.15.60:0/835603021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc50806fa30 0x7fc508129a10 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:02.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.781+0000 7f8adffff640 1 -- 172.21.15.60:0/939571867 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8ae8014410 con 0x7f8af812f110 2023-08-24T10:05:02.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.781+0000 7f8adffff640 1 -- 172.21.15.60:0/939571867 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f8ae8014570 con 0x7f8af812f110 2023-08-24T10:05:02.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.781+0000 7f8afdaec640 1 -- 172.21.15.60:0/939571867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af80a18e0 con 0x7f8af812f110 2023-08-24T10:05:02.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.781+0000 7fc5057fa640 1 -- 172.21.15.60:0/835603021 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc4fc014410 con 0x7fc50806fa30 2023-08-24T10:05:02.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.781+0000 7f8adffff640 1 -- 172.21.15.60:0/939571867 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8ae8014810 con 0x7f8af812f110 2023-08-24T10:05:02.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.781+0000 7fc5057fa640 1 -- 172.21.15.60:0/835603021 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc4fc014570 con 0x7fc50806fa30 2023-08-24T10:05:02.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.781+0000 7fc5057fa640 1 -- 172.21.15.60:0/835603021 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc4fc014810 con 0x7fc50806fa30 2023-08-24T10:05:02.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.781+0000 7fc50f19e640 1 -- 172.21.15.60:0/835603021 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5081308d0 con 0x7fc50806fa30 2023-08-24T10:05:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.781+0000 7fc50f19e640 1 -- 172.21.15.60:0/835603021 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50812a810 con 0x7fc50806fa30 2023-08-24T10:05:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.781+0000 7f8afdaec640 1 -- 172.21.15.60:0/939571867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af806a1d0 con 0x7f8af812f110 2023-08-24T10:05:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.781+0000 7f500feec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f500806d300 0x7f500806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.782+0000 7f500feec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500808d710 con 0x7f500806d300 2023-08-24T10:05:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.782+0000 7f500dc88640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f500806d300 0x7f500806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.782+0000 7f500dc88640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f500806d300 0x7f500806ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33112/0 (socket says 172.21.15.60:33112) 2023-08-24T10:05:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.782+0000 7f500dc88640 1 -- 172.21.15.60:0/4016793020 learned_addr learned my addr 172.21.15.60:0/4016793020 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:05:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.782+0000 7f148986c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f148406f3c0 0x7f14841c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.782+0000 7f148986c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14841c1cf0 con 0x7f148406f3c0 2023-08-24T10:05:02.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.782+0000 7f1482ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f148406f3c0 0x7f14841c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:02.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.782+0000 7f1482ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f148406f3c0 0x7f14841c1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33116/0 (socket says 172.21.15.60:33116) 2023-08-24T10:05:02.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.782+0000 7f1482ffd640 1 -- 172.21.15.60:0/335076166 learned_addr learned my addr 172.21.15.60:0/335076166 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:05:02.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.782+0000 7f500dc88640 1 -- 172.21.15.60:0/4016793020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500806ea80 con 0x7f500806d300 2023-08-24T10:05:02.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.782+0000 7f8afdaec640 1 -- 172.21.15.60:0/939571867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8af81334a0 con 0x7f8af812f110 2023-08-24T10:05:02.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.782+0000 7fc50f19e640 1 -- 172.21.15.60:0/835603021 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc508119f80 con 0x7fc50806fa30 2023-08-24T10:05:02.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.782+0000 7fc5057fa640 1 -- 172.21.15.60:0/835603021 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fc4fc014a10 con 0x7fc50806fa30 2023-08-24T10:05:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7f500dc88640 1 --2- 172.21.15.60:0/4016793020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f500806d300 0x7f500806ee40 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7f4ffe7fc640 1 -- 172.21.15.60:0/4016793020 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f500400fd70 con 0x7f500806d300 2023-08-24T10:05:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7f500feec640 1 -- 172.21.15.60:0/4016793020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500806e770 con 0x7f500806d300 2023-08-24T10:05:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7fc5057fa640 1 --2- 172.21.15.60:0/835603021 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc4ec0370e0 0x7fc4ec0395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7f4ffe7fc640 1 -- 172.21.15.60:0/4016793020 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f500400fed0 con 0x7f500806d300 2023-08-24T10:05:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7f500feec640 1 -- 172.21.15.60:0/4016793020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5008070f60 con 0x7f500806d300 2023-08-24T10:05:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7fc5057fa640 1 -- 172.21.15.60:0/835603021 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fc4fc019040 con 0x7fc50806fa30 2023-08-24T10:05:02.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7f4ffe7fc640 1 -- 172.21.15.60:0/4016793020 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5004011550 con 0x7f500806d300 2023-08-24T10:05:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7f1482ffd640 1 -- 172.21.15.60:0/335076166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148406e330 con 0x7f148406f3c0 2023-08-24T10:05:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7f1482ffd640 1 --2- 172.21.15.60:0/335076166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f148406f3c0 0x7f14841c1f90 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7fc5057fa640 1 --2- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 0x7fc4ec03e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7fc5057fa640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4ec03eba0 con 0x7fc4ec03c080 2023-08-24T10:05:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7fc5057fa640 1 -- 172.21.15.60:0/835603021 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (crc 0 0 0) 0x7fc4fc04a1b0 con 0x7fc50806fa30 2023-08-24T10:05:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7f8adffff640 1 -- 172.21.15.60:0/939571867 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f8ae80149c0 con 0x7f8af812f110 2023-08-24T10:05:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7fc50d73b640 1 -- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 msgr2=0x7fc4ec03e510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6802/2103781563 2023-08-24T10:05:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7fc50d73b640 1 --2- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 0x7fc4ec03e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:05:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7f8adffff640 1 --2- 172.21.15.60:0/939571867 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8ae4037180 0x7f8ae4039690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7f8adffff640 1 -- 172.21.15.60:0/939571867 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f8ae8019040 con 0x7f8af812f110 2023-08-24T10:05:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7f8adffff640 1 --2- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 0x7f8ae403e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7f8adffff640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ae403ec40 con 0x7f8ae403c120 2023-08-24T10:05:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7f8adffff640 1 -- 172.21.15.60:0/939571867 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (crc 0 0 0) 0x7f8ae804a050 con 0x7f8af812f110 2023-08-24T10:05:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7f146bfff640 1 -- 172.21.15.60:0/335076166 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1474014410 con 0x7f148406f3c0 2023-08-24T10:05:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7f146bfff640 1 -- 172.21.15.60:0/335076166 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1474014570 con 0x7f148406f3c0 2023-08-24T10:05:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7f146bfff640 1 -- 172.21.15.60:0/335076166 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1474014810 con 0x7f148406f3c0 2023-08-24T10:05:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7f148986c640 1 -- 172.21.15.60:0/335076166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148412abd0 con 0x7f148406f3c0 2023-08-24T10:05:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.783+0000 7f8af7fff640 1 -- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 msgr2=0x7f8ae403e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6802/3144688060 2023-08-24T10:05:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.784+0000 7f8af7fff640 1 --2- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 0x7f8ae403e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:05:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.784+0000 7f148986c640 1 -- 172.21.15.60:0/335076166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148412b8a0 con 0x7f148406f3c0 2023-08-24T10:05:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.784+0000 7fc507fff640 1 --2- 172.21.15.60:0/835603021 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc4ec0370e0 0x7fc4ec0395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.784+0000 7f500feec640 1 -- 172.21.15.60:0/4016793020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4ff0000f10 con 0x7f500806d300 2023-08-24T10:05:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.784+0000 7f8af6ffd640 1 --2- 172.21.15.60:0/939571867 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8ae4037180 0x7f8ae4039690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:02.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.784+0000 7fc507fff640 1 --2- 172.21.15.60:0/835603021 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc4ec0370e0 0x7fc4ec0395f0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:02.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.784+0000 7f4ffe7fc640 1 -- 172.21.15.60:0/4016793020 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f5004011700 con 0x7f500806d300 2023-08-24T10:05:02.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f148986c640 1 -- 172.21.15.60:0/335076166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1484130c40 con 0x7f148406f3c0 2023-08-24T10:05:02.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f8af6ffd640 1 --2- 172.21.15.60:0/939571867 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8ae4037180 0x7f8ae4039690 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:02.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f146bfff640 1 -- 172.21.15.60:0/335076166 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f1474014970 con 0x7f148406f3c0 2023-08-24T10:05:02.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f4ffe7fc640 1 --2- 172.21.15.60:0/4016793020 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4fec037180 0x7f4fec039690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f4ffe7fc640 1 -- 172.21.15.60:0/4016793020 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f5004017040 con 0x7f500806d300 2023-08-24T10:05:02.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f146bfff640 1 --2- 172.21.15.60:0/335076166 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1470037180 0x7f1470039690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f4ffe7fc640 1 --2- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 0x7f4fec03e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f4ffe7fc640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fec03ec40 con 0x7f4fec03c120 2023-08-24T10:05:02.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f4ffe7fc640 1 -- 172.21.15.60:0/4016793020 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (crc 0 0 0) 0x7f50040494e0 con 0x7f500806d300 2023-08-24T10:05:02.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f500d487640 1 --2- 172.21.15.60:0/4016793020 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4fec037180 0x7f4fec039690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:02.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f500e489640 1 -- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 msgr2=0x7f4fec03e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2841298624 2023-08-24T10:05:02.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f146bfff640 1 -- 172.21.15.60:0/335076166 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f1474019040 con 0x7f148406f3c0 2023-08-24T10:05:02.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f500e489640 1 --2- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 0x7f4fec03e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:05:02.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f146bfff640 1 --2- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 0x7f147003e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:05:02.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f146bfff640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f147003ec40 con 0x7f147003c120 2023-08-24T10:05:02.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f146bfff640 1 -- 172.21.15.60:0/335076166 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (crc 0 0 0) 0x7f1474049a00 con 0x7f148406f3c0 2023-08-24T10:05:02.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f14837fe640 1 -- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 msgr2=0x7f147003e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6804/1298459676 2023-08-24T10:05:02.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f14827fc640 1 --2- 172.21.15.60:0/335076166 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1470037180 0x7f1470039690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:05:02.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f14837fe640 1 --2- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 0x7f147003e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:05:02.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.785+0000 7f500d487640 1 --2- 172.21.15.60:0/4016793020 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4fec037180 0x7f4fec039690 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:02.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.786+0000 7f14827fc640 1 --2- 172.21.15.60:0/335076166 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1470037180 0x7f1470039690 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:05:02.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.984+0000 7fc50d73b640 1 -- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 msgr2=0x7fc4ec03e510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6802/2103781563 2023-08-24T10:05:02.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.984+0000 7fc50d73b640 1 --2- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 0x7fc4ec03e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:05:02.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.985+0000 7f8af7fff640 1 -- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 msgr2=0x7f8ae403e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6802/3144688060 2023-08-24T10:05:02.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.985+0000 7f8af7fff640 1 --2- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 0x7f8ae403e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:05:02.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.986+0000 7f500e489640 1 -- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 msgr2=0x7f4fec03e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2841298624 2023-08-24T10:05:02.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.986+0000 7f500e489640 1 --2- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 0x7f4fec03e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:05:02.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.986+0000 7f14837fe640 1 -- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 msgr2=0x7f147003e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6804/1298459676 2023-08-24T10:05:02.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:02.986+0000 7f14837fe640 1 --2- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 0x7f147003e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:05:03.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:03.385+0000 7fc50d73b640 1 -- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 msgr2=0x7fc4ec03e510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6802/2103781563 2023-08-24T10:05:03.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:03.385+0000 7fc50d73b640 1 --2- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 0x7fc4ec03e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:05:03.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:03.386+0000 7f8af7fff640 1 -- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 msgr2=0x7f8ae403e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6802/3144688060 2023-08-24T10:05:03.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:03.386+0000 7f8af7fff640 1 --2- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 0x7f8ae403e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:05:03.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:03.387+0000 7f500e489640 1 -- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 msgr2=0x7f4fec03e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2841298624 2023-08-24T10:05:03.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:03.387+0000 7f500e489640 1 --2- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 0x7f4fec03e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:05:03.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:03.387+0000 7f14837fe640 1 -- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 msgr2=0x7f147003e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6804/1298459676 2023-08-24T10:05:03.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:03.387+0000 7f14837fe640 1 --2- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 0x7f147003e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:05:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:04.186+0000 7fc50d73b640 1 -- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 msgr2=0x7fc4ec03e510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6802/2103781563 2023-08-24T10:05:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:04.186+0000 7fc50d73b640 1 --2- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 0x7fc4ec03e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:05:04.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:04.187+0000 7f8af7fff640 1 -- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 msgr2=0x7f8ae403e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6802/3144688060 2023-08-24T10:05:04.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:04.187+0000 7f8af7fff640 1 --2- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 0x7f8ae403e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:05:04.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:04.188+0000 7f500e489640 1 -- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 msgr2=0x7f4fec03e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2841298624 2023-08-24T10:05:04.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:04.188+0000 7f500e489640 1 --2- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 0x7f4fec03e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:05:04.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:04.188+0000 7f14837fe640 1 -- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 msgr2=0x7f147003e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6804/1298459676 2023-08-24T10:05:04.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:04.188+0000 7f14837fe640 1 --2- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 0x7f147003e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:05:05.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:05.788+0000 7fc50d73b640 1 -- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 msgr2=0x7fc4ec03e510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6802/2103781563 2023-08-24T10:05:05.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:05.788+0000 7fc50d73b640 1 --2- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 0x7fc4ec03e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-24T10:05:05.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:05.788+0000 7f8af7fff640 1 -- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 msgr2=0x7f8ae403e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6802/3144688060 2023-08-24T10:05:05.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:05.788+0000 7f8af7fff640 1 --2- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 0x7f8ae403e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-24T10:05:05.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:05.789+0000 7f500e489640 1 -- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 msgr2=0x7f4fec03e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2841298624 2023-08-24T10:05:05.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:05.789+0000 7f500e489640 1 --2- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 0x7f4fec03e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-24T10:05:05.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:05.790+0000 7f14837fe640 1 -- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 msgr2=0x7f147003e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6804/1298459676 2023-08-24T10:05:05.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:05.790+0000 7f14837fe640 1 --2- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 0x7f147003e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-08-24T10:05:07.452 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2023-08-24T10:05:07.453 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-08-24T10:05:07.453 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-08-24T10:05:07.453 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-08-24T10:05:07.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:07.782+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=15}) v3 -- 0x7fc4e40034a0 con 0x7fc50806fa30 2023-08-24T10:05:07.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:07.782+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e4003830 con 0x7fc4ec03c080 2023-08-24T10:05:07.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:07.782+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=15}) v3 -- 0x7f8ad40034a0 con 0x7f8af812f110 2023-08-24T10:05:07.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:07.782+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad4003830 con 0x7f8ae403c120 2023-08-24T10:05:07.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:07.784+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=15}) v3 -- 0x7f4fe80034a0 con 0x7f500806d300 2023-08-24T10:05:07.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:07.784+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe8003830 con 0x7f4fec03c120 2023-08-24T10:05:07.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:07.785+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=15}) v3 -- 0x7f14600034a0 con 0x7f148406f3c0 2023-08-24T10:05:07.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:07.785+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f1460003830 con 0x7f147003c120 2023-08-24T10:05:08.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:08.991+0000 7fc50d73b640 1 -- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 msgr2=0x7fc4ec03e510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6802/2103781563 2023-08-24T10:05:08.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:08.991+0000 7fc50d73b640 1 --2- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 0x7fc4ec03e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-08-24T10:05:08.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:08.991+0000 7f8af7fff640 1 -- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 msgr2=0x7f8ae403e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6802/3144688060 2023-08-24T10:05:08.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:08.991+0000 7f8af7fff640 1 --2- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 0x7f8ae403e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-08-24T10:05:08.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:08.993+0000 7f500e489640 1 -- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 msgr2=0x7f4fec03e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2841298624 2023-08-24T10:05:08.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:08.993+0000 7f500e489640 1 --2- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 0x7f4fec03e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-08-24T10:05:08.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:08.993+0000 7f14837fe640 1 -- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 msgr2=0x7f147003e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6804/1298459676 2023-08-24T10:05:08.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:08.993+0000 7f14837fe640 1 --2- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 0x7f147003e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-08-24T10:05:12.655 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~5s 2023-08-24T10:05:12.655 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~5s 2023-08-24T10:05:12.656 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~5s 2023-08-24T10:05:12.656 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~5s 2023-08-24T10:05:12.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:12.783+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e40039a0 con 0x7fc4ec03c080 2023-08-24T10:05:12.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:12.783+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad40039a0 con 0x7f8ae403c120 2023-08-24T10:05:12.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:12.785+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe80039a0 con 0x7f4fec03c120 2023-08-24T10:05:12.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:12.786+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f14600039a0 con 0x7f147003c120 2023-08-24T10:05:15.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:15.398+0000 7fc50d73b640 1 -- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 msgr2=0x7fc4ec03e510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6802/2103781563 2023-08-24T10:05:15.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:15.398+0000 7fc50d73b640 1 --2- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 0x7fc4ec03e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2023-08-24T10:05:15.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:15.398+0000 7f8af7fff640 1 -- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 msgr2=0x7f8ae403e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6802/3144688060 2023-08-24T10:05:15.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:15.398+0000 7f8af7fff640 1 --2- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 0x7f8ae403e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2023-08-24T10:05:15.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:15.399+0000 7f500e489640 1 -- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 msgr2=0x7f4fec03e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2841298624 2023-08-24T10:05:15.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:15.399+0000 7f500e489640 1 --2- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 0x7f4fec03e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2023-08-24T10:05:15.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:15.400+0000 7f14837fe640 1 -- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 msgr2=0x7f147003e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6804/1298459676 2023-08-24T10:05:15.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:15.400+0000 7f14837fe640 1 --2- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 0x7f147003e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2023-08-24T10:05:17.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:17.784+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e4003b10 con 0x7fc4ec03c080 2023-08-24T10:05:17.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:17.784+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad4003b10 con 0x7f8ae403c120 2023-08-24T10:05:17.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:17.786+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe8003b10 con 0x7f4fec03c120 2023-08-24T10:05:17.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:17.787+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f1460003b10 con 0x7f147003c120 2023-08-24T10:05:17.858 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~10s 2023-08-24T10:05:17.859 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~10s 2023-08-24T10:05:17.859 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~10s 2023-08-24T10:05:17.859 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~10s 2023-08-24T10:05:22.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:22.785+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e4003c80 con 0x7fc4ec03c080 2023-08-24T10:05:22.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:22.785+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad4003c80 con 0x7f8ae403c120 2023-08-24T10:05:22.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:22.787+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe8003c80 con 0x7f4fec03c120 2023-08-24T10:05:22.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:22.788+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f1460003c80 con 0x7f147003c120 2023-08-24T10:05:23.061 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~16s 2023-08-24T10:05:23.061 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~16s 2023-08-24T10:05:23.062 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~16s 2023-08-24T10:05:23.062 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~16s 2023-08-24T10:05:27.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:27.786+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e4003df0 con 0x7fc4ec03c080 2023-08-24T10:05:27.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:27.786+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad4003df0 con 0x7f8ae403c120 2023-08-24T10:05:27.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:27.788+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe8003df0 con 0x7f4fec03c120 2023-08-24T10:05:27.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:27.789+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f1460003df0 con 0x7f147003c120 2023-08-24T10:05:28.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:28.202+0000 7f500e489640 1 -- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 msgr2=0x7f4fec03e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2841298624 2023-08-24T10:05:28.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:28.202+0000 7f500e489640 1 --2- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 0x7f4fec03e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:05:28.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:28.202+0000 7f14837fe640 1 -- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 msgr2=0x7f147003e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6804/1298459676 2023-08-24T10:05:28.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:28.202+0000 7f14837fe640 1 --2- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 0x7f147003e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:05:28.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:28.209+0000 7fc50d73b640 1 -- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 msgr2=0x7fc4ec03e510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6802/2103781563 2023-08-24T10:05:28.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:28.209+0000 7fc50d73b640 1 --2- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 0x7fc4ec03e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:05:28.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:28.211+0000 7f8af7fff640 1 -- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 msgr2=0x7f8ae403e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6802/3144688060 2023-08-24T10:05:28.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:28.211+0000 7f8af7fff640 1 --2- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 0x7f8ae403e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:05:28.263 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~21s 2023-08-24T10:05:28.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~21s 2023-08-24T10:05:28.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~21s 2023-08-24T10:05:28.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~21s 2023-08-24T10:05:32.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:32.787+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e4003f60 con 0x7fc4ec03c080 2023-08-24T10:05:32.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:32.787+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad4003f60 con 0x7f8ae403c120 2023-08-24T10:05:32.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:32.789+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe8003f60 con 0x7f4fec03c120 2023-08-24T10:05:32.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:32.790+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f1460003f60 con 0x7f147003c120 2023-08-24T10:05:33.466 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~26s 2023-08-24T10:05:33.467 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~26s 2023-08-24T10:05:33.467 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~26s 2023-08-24T10:05:33.467 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~26s 2023-08-24T10:05:37.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:37.788+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e40040d0 con 0x7fc4ec03c080 2023-08-24T10:05:37.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:37.788+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad40040d0 con 0x7f8ae403c120 2023-08-24T10:05:37.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:37.790+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe80040d0 con 0x7f4fec03c120 2023-08-24T10:05:37.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:37.791+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f14600040d0 con 0x7f147003c120 2023-08-24T10:05:38.669 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~31s 2023-08-24T10:05:38.670 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~31s 2023-08-24T10:05:38.670 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~31s 2023-08-24T10:05:38.670 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~31s 2023-08-24T10:05:42.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:42.789+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e4004240 con 0x7fc4ec03c080 2023-08-24T10:05:42.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:42.789+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad4004240 con 0x7f8ae403c120 2023-08-24T10:05:42.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:42.791+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe8004240 con 0x7f4fec03c120 2023-08-24T10:05:42.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:42.792+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f1460004240 con 0x7f147003c120 2023-08-24T10:05:43.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:43.210+0000 7f500e489640 1 -- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 msgr2=0x7f4fec03e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2841298624 2023-08-24T10:05:43.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:43.210+0000 7f500e489640 1 --2- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 0x7f4fec03e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:05:43.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:43.217+0000 7f14837fe640 1 -- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 msgr2=0x7f147003e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6804/1298459676 2023-08-24T10:05:43.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:43.217+0000 7f14837fe640 1 --2- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 0x7f147003e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:05:43.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:43.224+0000 7fc50d73b640 1 -- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 msgr2=0x7fc4ec03e510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6802/2103781563 2023-08-24T10:05:43.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:43.224+0000 7fc50d73b640 1 --2- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 0x7fc4ec03e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:05:43.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:43.226+0000 7f8af7fff640 1 -- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 msgr2=0x7f8ae403e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6802/3144688060 2023-08-24T10:05:43.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:43.226+0000 7f8af7fff640 1 --2- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 0x7f8ae403e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:05:43.873 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~36s 2023-08-24T10:05:43.873 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~36s 2023-08-24T10:05:43.873 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~36s 2023-08-24T10:05:43.873 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~36s 2023-08-24T10:05:47.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:47.789+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e40043b0 con 0x7fc4ec03c080 2023-08-24T10:05:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:47.789+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad40043b0 con 0x7f8ae403c120 2023-08-24T10:05:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:47.791+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe80043b0 con 0x7f4fec03c120 2023-08-24T10:05:47.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:47.792+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f14600043b0 con 0x7f147003c120 2023-08-24T10:05:49.076 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~42s 2023-08-24T10:05:49.077 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~42s 2023-08-24T10:05:49.077 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~42s 2023-08-24T10:05:49.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~42s 2023-08-24T10:05:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:52.790+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e4004520 con 0x7fc4ec03c080 2023-08-24T10:05:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:52.790+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad4004520 con 0x7f8ae403c120 2023-08-24T10:05:52.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:52.792+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe8004520 con 0x7f4fec03c120 2023-08-24T10:05:52.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:52.793+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f1460004520 con 0x7f147003c120 2023-08-24T10:05:54.281 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~47s 2023-08-24T10:05:54.281 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~47s 2023-08-24T10:05:54.281 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~47s 2023-08-24T10:05:54.281 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~47s 2023-08-24T10:05:57.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:57.791+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e4004690 con 0x7fc4ec03c080 2023-08-24T10:05:57.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:57.791+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad4004690 con 0x7f8ae403c120 2023-08-24T10:05:57.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:57.793+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe8004690 con 0x7f4fec03c120 2023-08-24T10:05:57.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:57.794+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f1460004690 con 0x7f147003c120 2023-08-24T10:05:58.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:58.225+0000 7f500e489640 1 -- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 msgr2=0x7f4fec03e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2841298624 2023-08-24T10:05:58.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:58.225+0000 7f500e489640 1 --2- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 0x7f4fec03e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:05:58.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:58.229+0000 7f14837fe640 1 -- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 msgr2=0x7f147003e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6804/1298459676 2023-08-24T10:05:58.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:58.229+0000 7f14837fe640 1 --2- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 0x7f147003e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:05:58.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:58.238+0000 7fc50d73b640 1 -- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 msgr2=0x7fc4ec03e510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6802/2103781563 2023-08-24T10:05:58.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:58.238+0000 7fc50d73b640 1 --2- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 0x7fc4ec03e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:05:58.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:58.241+0000 7f8af7fff640 1 -- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 msgr2=0x7f8ae403e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6802/3144688060 2023-08-24T10:05:58.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:05:58.241+0000 7f8af7fff640 1 --2- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 0x7f8ae403e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:05:59.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~52s 2023-08-24T10:05:59.487 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~52s 2023-08-24T10:05:59.487 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~52s 2023-08-24T10:05:59.487 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~52s 2023-08-24T10:06:02.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:02.792+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e4004800 con 0x7fc4ec03c080 2023-08-24T10:06:02.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:02.792+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad4004800 con 0x7f8ae403c120 2023-08-24T10:06:02.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:02.794+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe8004800 con 0x7f4fec03c120 2023-08-24T10:06:02.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:02.795+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f1460004800 con 0x7f147003c120 2023-08-24T10:06:04.689 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~57s 2023-08-24T10:06:04.689 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~57s 2023-08-24T10:06:04.690 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~57s 2023-08-24T10:06:04.690 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~57s 2023-08-24T10:06:07.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:07.793+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e4004970 con 0x7fc4ec03c080 2023-08-24T10:06:07.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:07.793+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad4004970 con 0x7f8ae403c120 2023-08-24T10:06:07.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:07.795+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe8004970 con 0x7f4fec03c120 2023-08-24T10:06:07.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:07.796+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f1460004970 con 0x7f147003c120 2023-08-24T10:06:09.892 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~62s 2023-08-24T10:06:09.893 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~62s 2023-08-24T10:06:09.893 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~62s 2023-08-24T10:06:09.893 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~62s 2023-08-24T10:06:12.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:12.794+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e4004ae0 con 0x7fc4ec03c080 2023-08-24T10:06:12.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:12.794+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad4004ae0 con 0x7f8ae403c120 2023-08-24T10:06:12.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:12.796+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe8004ae0 con 0x7f4fec03c120 2023-08-24T10:06:12.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:12.797+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f1460004ae0 con 0x7f147003c120 2023-08-24T10:06:13.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:13.234+0000 7f14837fe640 1 -- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 msgr2=0x7f147003e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6804/1298459676 2023-08-24T10:06:13.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:13.234+0000 7f14837fe640 1 --2- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 0x7f147003e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:06:13.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:13.235+0000 7f500e489640 1 -- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 msgr2=0x7f4fec03e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2841298624 2023-08-24T10:06:13.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:13.235+0000 7f500e489640 1 --2- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 0x7f4fec03e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:06:13.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:13.254+0000 7fc50d73b640 1 -- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 msgr2=0x7fc4ec03e510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6802/2103781563 2023-08-24T10:06:13.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:13.254+0000 7fc50d73b640 1 --2- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 0x7fc4ec03e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:06:13.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:13.257+0000 7f8af7fff640 1 -- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 msgr2=0x7f8ae403e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6802/3144688060 2023-08-24T10:06:13.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:13.257+0000 7f8af7fff640 1 --2- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 0x7f8ae403e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:06:15.097 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~68s 2023-08-24T10:06:15.097 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~68s 2023-08-24T10:06:15.097 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~68s 2023-08-24T10:06:15.097 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~68s 2023-08-24T10:06:17.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:17.795+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e4004c50 con 0x7fc4ec03c080 2023-08-24T10:06:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:17.795+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad4004c50 con 0x7f8ae403c120 2023-08-24T10:06:17.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:17.797+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe8004c50 con 0x7f4fec03c120 2023-08-24T10:06:17.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:17.798+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f1460004c50 con 0x7f147003c120 2023-08-24T10:06:20.300 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~73s 2023-08-24T10:06:20.301 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~73s 2023-08-24T10:06:20.301 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~73s 2023-08-24T10:06:20.301 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~73s 2023-08-24T10:06:22.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:22.796+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e4004dc0 con 0x7fc4ec03c080 2023-08-24T10:06:22.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:22.796+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad4004dc0 con 0x7f8ae403c120 2023-08-24T10:06:22.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:22.798+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe8004dc0 con 0x7f4fec03c120 2023-08-24T10:06:22.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:22.799+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f1460004dc0 con 0x7f147003c120 2023-08-24T10:06:25.503 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~78s 2023-08-24T10:06:25.503 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~78s 2023-08-24T10:06:25.503 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~78s 2023-08-24T10:06:25.503 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~78s 2023-08-24T10:06:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:27.797+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e4004f30 con 0x7fc4ec03c080 2023-08-24T10:06:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:27.797+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad4004f30 con 0x7f8ae403c120 2023-08-24T10:06:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:27.799+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe8004f30 con 0x7f4fec03c120 2023-08-24T10:06:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:27.800+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f1460004f30 con 0x7f147003c120 2023-08-24T10:06:28.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:28.249+0000 7f14837fe640 1 -- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 msgr2=0x7f147003e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6804/1298459676 2023-08-24T10:06:28.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:28.249+0000 7f14837fe640 1 --2- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 0x7f147003e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:06:28.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:28.250+0000 7f500e489640 1 -- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 msgr2=0x7f4fec03e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2841298624 2023-08-24T10:06:28.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:28.250+0000 7f500e489640 1 --2- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 0x7f4fec03e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:06:28.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:28.270+0000 7fc50d73b640 1 -- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 msgr2=0x7fc4ec03e510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6802/2103781563 2023-08-24T10:06:28.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:28.270+0000 7fc50d73b640 1 --2- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 0x7fc4ec03e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:06:28.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:28.272+0000 7f8af7fff640 1 -- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 msgr2=0x7f8ae403e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6802/3144688060 2023-08-24T10:06:28.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:28.272+0000 7f8af7fff640 1 --2- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 0x7f8ae403e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:06:30.708 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~83s 2023-08-24T10:06:30.708 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~83s 2023-08-24T10:06:30.708 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~83s 2023-08-24T10:06:30.708 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~83s 2023-08-24T10:06:32.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:32.798+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e40050a0 con 0x7fc4ec03c080 2023-08-24T10:06:32.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:32.798+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad40050a0 con 0x7f8ae403c120 2023-08-24T10:06:32.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:32.800+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe80050a0 con 0x7f4fec03c120 2023-08-24T10:06:32.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:32.801+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f14600050a0 con 0x7f147003c120 2023-08-24T10:06:35.912 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~88s 2023-08-24T10:06:35.912 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~88s 2023-08-24T10:06:35.912 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~88s 2023-08-24T10:06:35.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~88s 2023-08-24T10:06:37.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:37.799+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e4005210 con 0x7fc4ec03c080 2023-08-24T10:06:37.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:37.799+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad4005210 con 0x7f8ae403c120 2023-08-24T10:06:37.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:37.801+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe8005210 con 0x7f4fec03c120 2023-08-24T10:06:37.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:37.802+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f1460005210 con 0x7f147003c120 2023-08-24T10:06:41.115 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~94s 2023-08-24T10:06:41.115 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~94s 2023-08-24T10:06:41.115 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~94s 2023-08-24T10:06:41.115 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~94s 2023-08-24T10:06:42.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:42.800+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e4005380 con 0x7fc4ec03c080 2023-08-24T10:06:42.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:42.800+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad4005380 con 0x7f8ae403c120 2023-08-24T10:06:42.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:42.802+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe8005380 con 0x7f4fec03c120 2023-08-24T10:06:42.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:42.803+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f1460005380 con 0x7f147003c120 2023-08-24T10:06:43.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:43.265+0000 7f14837fe640 1 -- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 msgr2=0x7f147003e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6804/1298459676 2023-08-24T10:06:43.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:43.265+0000 7f14837fe640 1 --2- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 0x7f147003e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:06:43.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:43.265+0000 7f500e489640 1 -- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 msgr2=0x7f4fec03e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2841298624 2023-08-24T10:06:43.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:43.265+0000 7f500e489640 1 --2- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 0x7f4fec03e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:06:43.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:43.273+0000 7f8af7fff640 1 -- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 msgr2=0x7f8ae403e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6802/3144688060 2023-08-24T10:06:43.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:43.273+0000 7f8af7fff640 1 --2- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 0x7f8ae403e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:06:43.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:43.277+0000 7fc50d73b640 1 -- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 msgr2=0x7fc4ec03e510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6802/2103781563 2023-08-24T10:06:43.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:43.277+0000 7fc50d73b640 1 --2- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 0x7fc4ec03e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:06:46.318 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~99s 2023-08-24T10:06:46.318 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~99s 2023-08-24T10:06:46.319 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~99s 2023-08-24T10:06:46.319 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~99s 2023-08-24T10:06:47.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:47.801+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e40054f0 con 0x7fc4ec03c080 2023-08-24T10:06:47.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:47.801+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad40054f0 con 0x7f8ae403c120 2023-08-24T10:06:47.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:47.803+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe80054f0 con 0x7f4fec03c120 2023-08-24T10:06:47.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:47.804+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f14600054f0 con 0x7f147003c120 2023-08-24T10:06:51.522 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~104s 2023-08-24T10:06:51.523 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~104s 2023-08-24T10:06:51.523 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~104s 2023-08-24T10:06:51.523 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~104s 2023-08-24T10:06:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:52.802+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e4005660 con 0x7fc4ec03c080 2023-08-24T10:06:52.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:52.802+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad4005660 con 0x7f8ae403c120 2023-08-24T10:06:52.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:52.804+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe8005660 con 0x7f4fec03c120 2023-08-24T10:06:52.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:52.805+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f1460005660 con 0x7f147003c120 2023-08-24T10:06:56.727 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~109s 2023-08-24T10:06:56.728 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~109s 2023-08-24T10:06:56.728 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~109s 2023-08-24T10:06:56.728 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~109s 2023-08-24T10:06:57.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:57.803+0000 7fc505ffb640 1 -- 172.21.15.60:0/835603021 --> v2:172.21.15.60:6802/2103781563 -- ping magic: 0 v1 -- 0x7fc4e40057d0 con 0x7fc4ec03c080 2023-08-24T10:06:57.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:57.803+0000 7f8af4ff9640 1 -- 172.21.15.60:0/939571867 --> v2:172.21.15.135:6802/3144688060 -- ping magic: 0 v1 -- 0x7f8ad40057d0 con 0x7f8ae403c120 2023-08-24T10:06:57.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:57.805+0000 7f4ffeffd640 1 -- 172.21.15.60:0/4016793020 --> v2:172.21.15.60:6800/2841298624 -- ping magic: 0 v1 -- 0x7f4fe80057d0 con 0x7f4fec03c120 2023-08-24T10:06:57.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:57.806+0000 7f148886a640 1 -- 172.21.15.60:0/335076166 --> v2:172.21.15.135:6804/1298459676 -- ping magic: 0 v1 -- 0x7f14600057d0 con 0x7f147003c120 2023-08-24T10:06:58.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:58.266+0000 7f14837fe640 1 -- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 msgr2=0x7f147003e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6804/1298459676 2023-08-24T10:06:58.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:58.266+0000 7f14837fe640 1 --2- 172.21.15.60:0/335076166 >> v2:172.21.15.135:6804/1298459676 conn(0x7f147003c120 0x7f147003e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:06:58.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:58.271+0000 7f500e489640 1 -- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 msgr2=0x7f4fec03e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6800/2841298624 2023-08-24T10:06:58.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:58.271+0000 7f500e489640 1 --2- 172.21.15.60:0/4016793020 >> v2:172.21.15.60:6800/2841298624 conn(0x7f4fec03c120 0x7f4fec03e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:06:58.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:58.275+0000 7f8af7fff640 1 -- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 msgr2=0x7f8ae403e5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.135:6802/3144688060 2023-08-24T10:06:58.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:58.275+0000 7f8af7fff640 1 --2- 172.21.15.60:0/939571867 >> v2:172.21.15.135:6802/3144688060 conn(0x7f8ae403c120 0x7f8ae403e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:06:58.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:58.282+0000 7fc50d73b640 1 -- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 msgr2=0x7fc4ec03e510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.60:6802/2103781563 2023-08-24T10:06:58.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:06:58.282+0000 7fc50d73b640 1 --2- 172.21.15.60:0/835603021 >> v2:172.21.15.60:6802/2103781563 conn(0x7fc4ec03c080 0x7fc4ec03e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2023-08-24T10:07:01.932 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~114s 2023-08-24T10:07:01.932 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~114s 2023-08-24T10:07:01.933 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~114s 2023-08-24T10:07:01.933 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~114s 2023-08-24T10:07:02.691 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-08-24T10:07:02.691 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-08-24T10:07:02.692 INFO:tasks.ceph:ignoring flush pg stats error, probably testing upgrade: Command failed on smithi060 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats' 2023-08-24T10:07:02.693 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-08-24T10:07:02.693 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:02.695 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-08-24T10:07:02.695 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-08-24T10:07:02.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.851+0000 7f64f999e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:02.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.855+0000 7f64f999e640 1 Processor -- start 2023-08-24T10:07:02.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.855+0000 7f64f999e640 1 -- start start 2023-08-24T10:07:02.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.855+0000 7f64f999e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f64f4096860 0x7f64f4096c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:02.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.855+0000 7f64f999e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f40971b0 con 0x7f64f40990b0 2023-08-24T10:07:02.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.855+0000 7f64f999e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f406db70 con 0x7f64f4096860 2023-08-24T10:07:02.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.856+0000 7f64f2ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f64f4096860 0x7f64f4096c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:02.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.856+0000 7f64f2ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f64f4096860 0x7f64f4096c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43166/0 (socket says 172.21.15.60:43166) 2023-08-24T10:07:02.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.856+0000 7f64f2ffd640 1 -- 172.21.15.60:0/3337268515 learned_addr learned my addr 172.21.15.60:0/3337268515 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:02.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.856+0000 7f64f2ffd640 1 -- 172.21.15.60:0/3337268515 >> v1:172.21.15.60:6789/0 conn(0x7f64f40990b0 legacy=0x7f64f40994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:02.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.856+0000 7f64f2ffd640 1 -- 172.21.15.60:0/3337268515 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f406e320 con 0x7f64f4096860 2023-08-24T10:07:02.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.856+0000 7f64f2ffd640 1 --2- 172.21.15.60:0/3337268515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f4096860 0x7f64f4096c70 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab403a755e821f6 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:02.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.856+0000 7f64f1ffb640 1 -- 172.21.15.60:0/3337268515 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f64e4010570 con 0x7f64f4096860 2023-08-24T10:07:02.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.856+0000 7f64f1ffb640 1 -- 172.21.15.60:0/3337268515 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f64e40106d0 con 0x7f64f4096860 2023-08-24T10:07:02.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.856+0000 7f64f1ffb640 1 -- 172.21.15.60:0/3337268515 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f64e4010970 con 0x7f64f4096860 2023-08-24T10:07:02.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.857+0000 7f64f999e640 1 -- 172.21.15.60:0/3337268515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f4096860 msgr2=0x7f64f4096c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:02.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.857+0000 7f64f999e640 1 --2- 172.21.15.60:0/3337268515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f4096860 0x7f64f4096c70 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:02.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.857+0000 7f64f999e640 1 -- 172.21.15.60:0/3337268515 shutdown_connections 2023-08-24T10:07:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.857+0000 7f64f999e640 1 --2- 172.21.15.60:0/3337268515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f4096860 0x7f64f4096c70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.857+0000 7f64f999e640 1 -- 172.21.15.60:0/3337268515 >> 172.21.15.60:0/3337268515 conn(0x7f64f409e110 msgr2=0x7f64f409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.857+0000 7f64f999e640 1 -- 172.21.15.60:0/3337268515 shutdown_connections 2023-08-24T10:07:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.857+0000 7f64f999e640 1 -- 172.21.15.60:0/3337268515 wait complete. 2023-08-24T10:07:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.857+0000 7f64f999e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:02.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.858+0000 7f64f999e640 1 Processor -- start 2023-08-24T10:07:02.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.858+0000 7f64f999e640 1 -- start start 2023-08-24T10:07:02.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.859+0000 7f64f999e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f4096860 0x7f64f412c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:02.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.859+0000 7f64f999e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f406b150 con 0x7f64f4096860 2023-08-24T10:07:02.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.859+0000 7f64f2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f4096860 0x7f64f412c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:02.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.859+0000 7f64f2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f4096860 0x7f64f412c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43176/0 (socket says 172.21.15.60:43176) 2023-08-24T10:07:02.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.859+0000 7f64f2ffd640 1 -- 172.21.15.60:0/2183356156 learned_addr learned my addr 172.21.15.60:0/2183356156 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:02.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.859+0000 7f64f2ffd640 1 -- 172.21.15.60:0/2183356156 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f4130190 con 0x7f64f4096860 2023-08-24T10:07:02.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.860+0000 7f64f2ffd640 1 --2- 172.21.15.60:0/2183356156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f4096860 0x7f64f412c150 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:02.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.860+0000 7f64dbfff640 1 -- 172.21.15.60:0/2183356156 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f64e400f930 con 0x7f64f4096860 2023-08-24T10:07:02.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.860+0000 7f64dbfff640 1 -- 172.21.15.60:0/2183356156 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f64e400fa90 con 0x7f64f4096860 2023-08-24T10:07:02.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.860+0000 7f64f999e640 1 -- 172.21.15.60:0/2183356156 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f412ed10 con 0x7f64f4096860 2023-08-24T10:07:02.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.860+0000 7f64dbfff640 1 -- 172.21.15.60:0/2183356156 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f64e400fd30 con 0x7f64f4096860 2023-08-24T10:07:02.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.860+0000 7f64f999e640 1 -- 172.21.15.60:0/2183356156 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f4093e20 con 0x7f64f4096860 2023-08-24T10:07:02.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.861+0000 7f64dbfff640 1 -- 172.21.15.60:0/2183356156 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f64e400e360 con 0x7f64f4096860 2023-08-24T10:07:02.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.861+0000 7f64f999e640 1 -- 172.21.15.60:0/2183356156 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f412ef00 con 0x7f64f4096860 2023-08-24T10:07:02.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.861+0000 7f64dbfff640 1 --2- 172.21.15.60:0/2183356156 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f64e0037130 0x7f64e0039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:02.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.861+0000 7f64dbfff640 1 -- 172.21.15.60:0/2183356156 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f64e4016040 con 0x7f64f4096860 2023-08-24T10:07:02.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.861+0000 7f64f27fc640 1 --2- 172.21.15.60:0/2183356156 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f64e0037130 0x7f64e0039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:02.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.861+0000 7f64f27fc640 1 --2- 172.21.15.60:0/2183356156 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f64e0037130 0x7f64e0039640 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:02.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:02.866+0000 7f64dbfff640 1 -- 172.21.15.60:0/2183356156 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f64f412ef00 con 0x7f64f4096860 2023-08-24T10:07:03.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.053+0000 7f64f999e640 1 -- 172.21.15.60:0/2183356156 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f64f406e5a0 con 0x7f64e0037130 2023-08-24T10:07:03.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.055+0000 7f64dbfff640 1 -- 172.21.15.60:0/2183356156 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f64e8001650 con 0x7f64e0037130 2023-08-24T10:07:03.057 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:03.058 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:03.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.056+0000 7f64f999e640 1 -- 172.21.15.60:0/2183356156 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f64e0037130 msgr2=0x7f64e0039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:03.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.056+0000 7f64f999e640 1 --2- 172.21.15.60:0/2183356156 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f64e0037130 0x7f64e0039640 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:03.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.056+0000 7f64f999e640 1 -- 172.21.15.60:0/2183356156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f4096860 msgr2=0x7f64f412c150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:03.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.056+0000 7f64f999e640 1 --2- 172.21.15.60:0/2183356156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f4096860 0x7f64f412c150 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:03.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.056+0000 7f64f999e640 1 -- 172.21.15.60:0/2183356156 shutdown_connections 2023-08-24T10:07:03.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.056+0000 7f64f999e640 1 --2- 172.21.15.60:0/2183356156 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f64e0037130 0x7f64e0039640 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:03.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.056+0000 7f64f999e640 1 --2- 172.21.15.60:0/2183356156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64f4096860 0x7f64f412c150 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:03.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.056+0000 7f64f999e640 1 -- 172.21.15.60:0/2183356156 >> 172.21.15.60:0/2183356156 conn(0x7f64f409e110 msgr2=0x7f64f41b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:03.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.056+0000 7f64f999e640 1 -- 172.21.15.60:0/2183356156 shutdown_connections 2023-08-24T10:07:03.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.057+0000 7f64f999e640 1 -- 172.21.15.60:0/2183356156 wait complete. 2023-08-24T10:07:03.068 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-08-24T10:07:01.519544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:03.069 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:03.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.176+0000 7fb5a3d1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:03.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.180+0000 7fb5a3d1e640 1 Processor -- start 2023-08-24T10:07:03.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.180+0000 7fb5a3d1e640 1 -- start start 2023-08-24T10:07:03.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.181+0000 7fb5a3d1e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb59c131d40 0x7fb59c134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:03.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.181+0000 7fb5a3d1e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59c099440 con 0x7fb59c06d300 2023-08-24T10:07:03.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.181+0000 7fb5a3d1e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59c09a050 con 0x7fb59c131d40 2023-08-24T10:07:03.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.181+0000 7fb5a12b9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb59c131d40 0x7fb59c134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:03.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.181+0000 7fb5a12b9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb59c131d40 0x7fb59c134160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43186/0 (socket says 172.21.15.60:43186) 2023-08-24T10:07:03.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.181+0000 7fb5a12b9640 1 -- 172.21.15.60:0/3722780425 learned_addr learned my addr 172.21.15.60:0/3722780425 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:03.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.181+0000 7fb5a1aba640 1 --1- 172.21.15.60:0/3722780425 >> v1:172.21.15.60:6789/0 conn(0x7fb59c06d300 0x7fb59c12bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45320/0 (socket says 172.21.15.60:45320) 2023-08-24T10:07:03.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.181+0000 7fb5a12b9640 1 -- 172.21.15.60:0/3722780425 >> v1:172.21.15.60:6789/0 conn(0x7fb59c06d300 legacy=0x7fb59c12bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:03.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.181+0000 7fb5a12b9640 1 -- 172.21.15.60:0/3722780425 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c12b9d0 con 0x7fb59c131d40 2023-08-24T10:07:03.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.181+0000 7fb5a12b9640 1 --2- 172.21.15.60:0/3722780425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb59c131d40 0x7fb59c134160 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b4e527c390189e server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:03.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.181+0000 7fb5a0ab8640 1 -- 172.21.15.60:0/3722780425 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb58c00d580 con 0x7fb59c131d40 2023-08-24T10:07:03.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.181+0000 7fb5a0ab8640 1 -- 172.21.15.60:0/3722780425 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb58c00d6e0 con 0x7fb59c131d40 2023-08-24T10:07:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.181+0000 7fb5a0ab8640 1 -- 172.21.15.60:0/3722780425 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb58c00d980 con 0x7fb59c131d40 2023-08-24T10:07:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.182+0000 7fb5a3d1e640 1 -- 172.21.15.60:0/3722780425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb59c131d40 msgr2=0x7fb59c134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.182+0000 7fb5a3d1e640 1 --2- 172.21.15.60:0/3722780425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb59c131d40 0x7fb59c134160 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.182+0000 7fb5a3d1e640 1 -- 172.21.15.60:0/3722780425 shutdown_connections 2023-08-24T10:07:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.182+0000 7fb5a3d1e640 1 --2- 172.21.15.60:0/3722780425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb59c131d40 0x7fb59c134160 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.182+0000 7fb5a3d1e640 1 -- 172.21.15.60:0/3722780425 >> 172.21.15.60:0/3722780425 conn(0x7fb59c09e110 msgr2=0x7fb59c09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.182+0000 7fb5a3d1e640 1 -- 172.21.15.60:0/3722780425 shutdown_connections 2023-08-24T10:07:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.182+0000 7fb5a3d1e640 1 -- 172.21.15.60:0/3722780425 wait complete. 2023-08-24T10:07:03.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.182+0000 7fb5a3d1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:03.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.183+0000 7fb5a3d1e640 1 Processor -- start 2023-08-24T10:07:03.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.183+0000 7fb5a3d1e640 1 -- start start 2023-08-24T10:07:03.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.184+0000 7fb5a3d1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb59c06d300 0x7fb59c0968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:03.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.184+0000 7fb5a3d1e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c071410 con 0x7fb59c06d300 2023-08-24T10:07:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.184+0000 7fb5a1aba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb59c06d300 0x7fb59c0968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.184+0000 7fb5a1aba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb59c06d300 0x7fb59c0968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43194/0 (socket says 172.21.15.60:43194) 2023-08-24T10:07:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.184+0000 7fb5a1aba640 1 -- 172.21.15.60:0/1964975935 learned_addr learned my addr 172.21.15.60:0/1964975935 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.184+0000 7fb5a1aba640 1 -- 172.21.15.60:0/1964975935 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c06f480 con 0x7fb59c06d300 2023-08-24T10:07:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.184+0000 7fb5a1aba640 1 --2- 172.21.15.60:0/1964975935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb59c06d300 0x7fb59c0968d0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.185+0000 7fb5927fc640 1 -- 172.21.15.60:0/1964975935 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb5980071e0 con 0x7fb59c06d300 2023-08-24T10:07:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.185+0000 7fb5927fc640 1 -- 172.21.15.60:0/1964975935 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb598007340 con 0x7fb59c06d300 2023-08-24T10:07:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.185+0000 7fb5a3d1e640 1 -- 172.21.15.60:0/1964975935 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59c071810 con 0x7fb59c06d300 2023-08-24T10:07:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.185+0000 7fb5927fc640 1 -- 172.21.15.60:0/1964975935 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb5980075e0 con 0x7fb59c06d300 2023-08-24T10:07:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.185+0000 7fb5a3d1e640 1 -- 172.21.15.60:0/1964975935 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59c06e2d0 con 0x7fb59c06d300 2023-08-24T10:07:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.185+0000 7fb5927fc640 1 -- 172.21.15.60:0/1964975935 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fb5980057e0 con 0x7fb59c06d300 2023-08-24T10:07:03.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.185+0000 7fb5a3d1e640 1 -- 172.21.15.60:0/1964975935 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59c06d9f0 con 0x7fb59c06d300 2023-08-24T10:07:03.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.186+0000 7fb5927fc640 1 --2- 172.21.15.60:0/1964975935 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb580037130 0x7fb580039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:03.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.186+0000 7fb5927fc640 1 -- 172.21.15.60:0/1964975935 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fb598046370 con 0x7fb59c06d300 2023-08-24T10:07:03.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.186+0000 7fb5a12b9640 1 --2- 172.21.15.60:0/1964975935 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb580037130 0x7fb580039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:03.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.186+0000 7fb5a12b9640 1 --2- 172.21.15.60:0/1964975935 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb580037130 0x7fb580039640 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.190+0000 7fb5927fc640 1 -- 172.21.15.60:0/1964975935 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fb59800d330 con 0x7fb59c06d300 2023-08-24T10:07:03.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.362+0000 7fb5a3d1e640 1 -- 172.21.15.60:0/1964975935 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb59c06fc00 con 0x7fb580037130 2023-08-24T10:07:03.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.363+0000 7fb5927fc640 1 -- 172.21.15.60:0/1964975935 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7fb59c06fc00 con 0x7fb580037130 2023-08-24T10:07:03.365 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:03.365 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:03.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.365+0000 7fb5a3d1e640 1 -- 172.21.15.60:0/1964975935 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb580037130 msgr2=0x7fb580039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:03.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.365+0000 7fb5a3d1e640 1 --2- 172.21.15.60:0/1964975935 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb580037130 0x7fb580039640 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:03.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.365+0000 7fb5a3d1e640 1 -- 172.21.15.60:0/1964975935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb59c06d300 msgr2=0x7fb59c0968d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:03.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.365+0000 7fb5a3d1e640 1 --2- 172.21.15.60:0/1964975935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb59c06d300 0x7fb59c0968d0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:03.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.365+0000 7fb5a3d1e640 1 -- 172.21.15.60:0/1964975935 shutdown_connections 2023-08-24T10:07:03.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.365+0000 7fb5a3d1e640 1 --2- 172.21.15.60:0/1964975935 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb580037130 0x7fb580039640 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:03.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.365+0000 7fb5a3d1e640 1 --2- 172.21.15.60:0/1964975935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb59c06d300 0x7fb59c0968d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:03.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.365+0000 7fb5a3d1e640 1 -- 172.21.15.60:0/1964975935 >> 172.21.15.60:0/1964975935 conn(0x7fb59c09e110 msgr2=0x7fb59c12ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:03.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.365+0000 7fb5a3d1e640 1 -- 172.21.15.60:0/1964975935 shutdown_connections 2023-08-24T10:07:03.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.365+0000 7fb5a3d1e640 1 -- 172.21.15.60:0/1964975935 wait complete. 2023-08-24T10:07:03.377 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-08-24T10:07:01.519544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:03.378 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:03.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.485+0000 7fb38f59e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:03.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.489+0000 7fb38f59e640 1 Processor -- start 2023-08-24T10:07:03.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.489+0000 7fb38f59e640 1 -- start start 2023-08-24T10:07:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.490+0000 7fb38f59e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb38812bc80 0x7fb38812c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.490+0000 7fb38f59e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb388099440 con 0x7fb38806d300 2023-08-24T10:07:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.490+0000 7fb38f59e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb388097e20 con 0x7fb38812bc80 2023-08-24T10:07:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.490+0000 7fb38d33a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb38806d300 0x7fb388097710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45336/0 (socket says 172.21.15.60:45336) 2023-08-24T10:07:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.490+0000 7fb38d33a640 1 -- 172.21.15.60:0/2006709746 learned_addr learned my addr 172.21.15.60:0/2006709746 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:07:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.490+0000 7fb38cb39640 1 --2- 172.21.15.60:0/2006709746 >> v2:172.21.15.60:3300/0 conn(0x7fb38812bc80 0x7fb38812c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.490+0000 7fb38cb39640 1 -- 172.21.15.60:0/2006709746 >> v1:172.21.15.60:6789/0 conn(0x7fb38806d300 legacy=0x7fb388097710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:03.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.490+0000 7fb38cb39640 1 -- 172.21.15.60:0/2006709746 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38809a050 con 0x7fb38812bc80 2023-08-24T10:07:03.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.491+0000 7fb38cb39640 1 --2- 172.21.15.60:0/2006709746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb38812bc80 0x7fb38812c090 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5835da6d0e8a0b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:03.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.491+0000 7fb387fff640 1 -- 172.21.15.60:0/2006709746 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb37c009ce0 con 0x7fb38812bc80 2023-08-24T10:07:03.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.491+0000 7fb387fff640 1 -- 172.21.15.60:0/2006709746 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb37c009e40 con 0x7fb38812bc80 2023-08-24T10:07:03.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.491+0000 7fb387fff640 1 -- 172.21.15.60:0/2006709746 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb37c00e360 con 0x7fb38812bc80 2023-08-24T10:07:03.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.491+0000 7fb38f59e640 1 -- 172.21.15.60:0/2006709746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb38812bc80 msgr2=0x7fb38812c090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:03.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.491+0000 7fb38f59e640 1 --2- 172.21.15.60:0/2006709746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb38812bc80 0x7fb38812c090 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:03.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.491+0000 7fb38f59e640 1 -- 172.21.15.60:0/2006709746 shutdown_connections 2023-08-24T10:07:03.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.491+0000 7fb38f59e640 1 --2- 172.21.15.60:0/2006709746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb38812bc80 0x7fb38812c090 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:03.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.492+0000 7fb38f59e640 1 -- 172.21.15.60:0/2006709746 >> 172.21.15.60:0/2006709746 conn(0x7fb38809e110 msgr2=0x7fb38809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:03.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.492+0000 7fb38f59e640 1 -- 172.21.15.60:0/2006709746 shutdown_connections 2023-08-24T10:07:03.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.492+0000 7fb38f59e640 1 -- 172.21.15.60:0/2006709746 wait complete. 2023-08-24T10:07:03.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.492+0000 7fb38f59e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:03.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.493+0000 7fb38f59e640 1 Processor -- start 2023-08-24T10:07:03.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.493+0000 7fb38f59e640 1 -- start start 2023-08-24T10:07:03.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.493+0000 7fb38f59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb38806d300 0x7fb3881c3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:03.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.493+0000 7fb38f59e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb388129b80 con 0x7fb38806d300 2023-08-24T10:07:03.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.494+0000 7fb38d33a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb38806d300 0x7fb3881c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:03.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.494+0000 7fb38d33a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb38806d300 0x7fb3881c3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43212/0 (socket says 172.21.15.60:43212) 2023-08-24T10:07:03.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.494+0000 7fb38d33a640 1 -- 172.21.15.60:0/3433743299 learned_addr learned my addr 172.21.15.60:0/3433743299 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:03.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.494+0000 7fb38d33a640 1 -- 172.21.15.60:0/3433743299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb388096f00 con 0x7fb38806d300 2023-08-24T10:07:03.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.494+0000 7fb38d33a640 1 --2- 172.21.15.60:0/3433743299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb38806d300 0x7fb3881c3140 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:03.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.494+0000 7fb385ffb640 1 -- 172.21.15.60:0/3433743299 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb3780070b0 con 0x7fb38806d300 2023-08-24T10:07:03.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.494+0000 7fb385ffb640 1 -- 172.21.15.60:0/3433743299 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb378007210 con 0x7fb38806d300 2023-08-24T10:07:03.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.494+0000 7fb38f59e640 1 -- 172.21.15.60:0/3433743299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb388089440 con 0x7fb38806d300 2023-08-24T10:07:03.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.494+0000 7fb385ffb640 1 -- 172.21.15.60:0/3433743299 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb3780074b0 con 0x7fb38806d300 2023-08-24T10:07:03.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.494+0000 7fb38f59e640 1 -- 172.21.15.60:0/3433743299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb388088ff0 con 0x7fb38806d300 2023-08-24T10:07:03.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.495+0000 7fb385ffb640 1 -- 172.21.15.60:0/3433743299 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fb3780076a0 con 0x7fb38806d300 2023-08-24T10:07:03.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.495+0000 7fb385ffb640 1 --2- 172.21.15.60:0/3433743299 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb374037130 0x7fb374039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:03.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.495+0000 7fb385ffb640 1 -- 172.21.15.60:0/3433743299 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fb378046ef0 con 0x7fb38806d300 2023-08-24T10:07:03.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.495+0000 7fb38f59e640 1 -- 172.21.15.60:0/3433743299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3881283f0 con 0x7fb38806d300 2023-08-24T10:07:03.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.496+0000 7fb38cb39640 1 --2- 172.21.15.60:0/3433743299 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb374037130 0x7fb374039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:03.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.496+0000 7fb38cb39640 1 --2- 172.21.15.60:0/3433743299 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb374037130 0x7fb374039640 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:03.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.500+0000 7fb385ffb640 1 -- 172.21.15.60:0/3433743299 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fb378073990 con 0x7fb38806d300 2023-08-24T10:07:03.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.675+0000 7fb38f59e640 1 -- 172.21.15.60:0/3433743299 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb38806d710 con 0x7fb374037130 2023-08-24T10:07:03.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.677+0000 7fb385ffb640 1 -- 172.21.15.60:0/3433743299 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7fb37c0125b0 con 0x7fb374037130 2023-08-24T10:07:03.678 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:03.678 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:03.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.679+0000 7fb38f59e640 1 -- 172.21.15.60:0/3433743299 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb374037130 msgr2=0x7fb374039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:03.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.679+0000 7fb38f59e640 1 --2- 172.21.15.60:0/3433743299 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb374037130 0x7fb374039640 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:03.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.679+0000 7fb38f59e640 1 -- 172.21.15.60:0/3433743299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb38806d300 msgr2=0x7fb3881c3140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:03.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.679+0000 7fb38f59e640 1 --2- 172.21.15.60:0/3433743299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb38806d300 0x7fb3881c3140 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:03.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.679+0000 7fb38f59e640 1 -- 172.21.15.60:0/3433743299 shutdown_connections 2023-08-24T10:07:03.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.679+0000 7fb38f59e640 1 --2- 172.21.15.60:0/3433743299 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb374037130 0x7fb374039640 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:03.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.679+0000 7fb38f59e640 1 --2- 172.21.15.60:0/3433743299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb38806d300 0x7fb3881c3140 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:03.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.679+0000 7fb38f59e640 1 -- 172.21.15.60:0/3433743299 >> 172.21.15.60:0/3433743299 conn(0x7fb38809e110 msgr2=0x7fb38812e820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:03.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.679+0000 7fb38f59e640 1 -- 172.21.15.60:0/3433743299 shutdown_connections 2023-08-24T10:07:03.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.679+0000 7fb38f59e640 1 -- 172.21.15.60:0/3433743299 wait complete. 2023-08-24T10:07:03.690 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2023-08-24T10:07:03.519856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:03.691 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:07:03.692 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:03.692 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:07:03.692 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:03.693 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:07:03.693 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:03.693 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:07:03.693 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:03.693 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:07:03.694 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:03.694 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:07:03.694 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:03.694 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:07:03.695 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:03.695 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:07:03.695 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:03.695 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:07:03.696 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:03.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:07:03.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.803+0000 7f04f8430640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.806+0000 7f04f8430640 1 Processor -- start 2023-08-24T10:07:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.806+0000 7f04f8430640 1 -- start start 2023-08-24T10:07:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.807+0000 7f04f8430640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f04f012f610 0x7f04f012fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.807+0000 7f04f8430640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04f009d750 con 0x7f04f01326f0 2023-08-24T10:07:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.807+0000 7f04f8430640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04f009d980 con 0x7f04f012f610 2023-08-24T10:07:03.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.807+0000 7f04f61cc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f04f012f610 0x7f04f012fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:03.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.807+0000 7f04f61cc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f04f012f610 0x7f04f012fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43216/0 (socket says 172.21.15.60:43216) 2023-08-24T10:07:03.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.807+0000 7f04f61cc640 1 -- 172.21.15.60:0/3225017914 learned_addr learned my addr 172.21.15.60:0/3225017914 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:03.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.808+0000 7f04f61cc640 1 -- 172.21.15.60:0/3225017914 >> v1:172.21.15.60:6789/0 conn(0x7f04f01326f0 legacy=0x7f04f0132b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:03.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.808+0000 7f04f61cc640 1 -- 172.21.15.60:0/3225017914 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f009bde0 con 0x7f04f012f610 2023-08-24T10:07:03.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.808+0000 7f04f61cc640 1 --2- 172.21.15.60:0/3225017914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04f012f610 0x7f04f012fa20 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ee1eb2988d44a7 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:03.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.808+0000 7f04f51ca640 1 -- 172.21.15.60:0/3225017914 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f04ec009dc0 con 0x7f04f012f610 2023-08-24T10:07:03.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.808+0000 7f04f51ca640 1 -- 172.21.15.60:0/3225017914 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f04ec00e230 con 0x7f04f012f610 2023-08-24T10:07:03.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.808+0000 7f04f51ca640 1 -- 172.21.15.60:0/3225017914 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f04ec00e4d0 con 0x7f04f012f610 2023-08-24T10:07:03.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.809+0000 7f04f8430640 1 -- 172.21.15.60:0/3225017914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04f012f610 msgr2=0x7f04f012fa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:03.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.809+0000 7f04f8430640 1 --2- 172.21.15.60:0/3225017914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04f012f610 0x7f04f012fa20 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:03.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.809+0000 7f04f8430640 1 -- 172.21.15.60:0/3225017914 shutdown_connections 2023-08-24T10:07:03.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.809+0000 7f04f8430640 1 --2- 172.21.15.60:0/3225017914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04f012f610 0x7f04f012fa20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.809+0000 7f04f8430640 1 -- 172.21.15.60:0/3225017914 >> 172.21.15.60:0/3225017914 conn(0x7f04f00a2500 msgr2=0x7f04f00a2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.809+0000 7f04f8430640 1 -- 172.21.15.60:0/3225017914 shutdown_connections 2023-08-24T10:07:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.809+0000 7f04f8430640 1 -- 172.21.15.60:0/3225017914 wait complete. 2023-08-24T10:07:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.809+0000 7f04f8430640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.810+0000 7f04f8430640 1 Processor -- start 2023-08-24T10:07:03.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.810+0000 7f04f8430640 1 -- start start 2023-08-24T10:07:03.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.811+0000 7f04f8430640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04f012f610 0x7f04f009fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:03.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.811+0000 7f04f8430640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f012cc60 con 0x7f04f012f610 2023-08-24T10:07:03.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.811+0000 7f04f61cc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04f012f610 0x7f04f009fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:03.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.811+0000 7f04f61cc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04f012f610 0x7f04f009fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43226/0 (socket says 172.21.15.60:43226) 2023-08-24T10:07:03.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.811+0000 7f04f61cc640 1 -- 172.21.15.60:0/3706359203 learned_addr learned my addr 172.21.15.60:0/3706359203 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:03.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.811+0000 7f04f61cc640 1 -- 172.21.15.60:0/3706359203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f006a1f0 con 0x7f04f012f610 2023-08-24T10:07:03.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.811+0000 7f04f61cc640 1 --2- 172.21.15.60:0/3706359203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04f012f610 0x7f04f009fca0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:03.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.811+0000 7f04e6ffd640 1 -- 172.21.15.60:0/3706359203 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f04ec014540 con 0x7f04f012f610 2023-08-24T10:07:03.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.811+0000 7f04e6ffd640 1 -- 172.21.15.60:0/3706359203 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f04ec0146a0 con 0x7f04f012f610 2023-08-24T10:07:03.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.811+0000 7f04e6ffd640 1 -- 172.21.15.60:0/3706359203 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f04ec014940 con 0x7f04f012f610 2023-08-24T10:07:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.811+0000 7f04f8430640 1 -- 172.21.15.60:0/3706359203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f009f7e0 con 0x7f04f012f610 2023-08-24T10:07:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.811+0000 7f04f8430640 1 -- 172.21.15.60:0/3706359203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f009f4d0 con 0x7f04f012f610 2023-08-24T10:07:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.812+0000 7f04e6ffd640 1 -- 172.21.15.60:0/3706359203 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f04ec014af0 con 0x7f04f012f610 2023-08-24T10:07:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.812+0000 7f04f8430640 1 -- 172.21.15.60:0/3706359203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f0061800 con 0x7f04f012f610 2023-08-24T10:07:03.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.812+0000 7f04e6ffd640 1 --2- 172.21.15.60:0/3706359203 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f04d4037130 0x7f04d4039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:03.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.812+0000 7f04e6ffd640 1 -- 172.21.15.60:0/3706359203 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f04ec019040 con 0x7f04f012f610 2023-08-24T10:07:03.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.813+0000 7f04f59cb640 1 --2- 172.21.15.60:0/3706359203 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f04d4037130 0x7f04d4039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:03.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.813+0000 7f04f59cb640 1 --2- 172.21.15.60:0/3706359203 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f04d4037130 0x7f04d4039640 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:03.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:03.817+0000 7f04e6ffd640 1 -- 172.21.15.60:0/3706359203 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f04ec00fb00 con 0x7f04f012f610 2023-08-24T10:07:04.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.059+0000 7f04f8430640 1 -- 172.21.15.60:0/3706359203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f04f006a1f0 con 0x7f04f012f610 2023-08-24T10:07:04.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.060+0000 7f04e6ffd640 1 -- 172.21.15.60:0/3706359203 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f04f006a1f0 con 0x7f04f012f610 2023-08-24T10:07:04.061 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:04.061 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":134,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:07:04.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.062+0000 7f04f8430640 1 -- 172.21.15.60:0/3706359203 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f04d4037130 msgr2=0x7f04d4039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:04.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.062+0000 7f04f8430640 1 --2- 172.21.15.60:0/3706359203 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f04d4037130 0x7f04d4039640 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:04.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.062+0000 7f04f8430640 1 -- 172.21.15.60:0/3706359203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04f012f610 msgr2=0x7f04f009fca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:04.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.062+0000 7f04f8430640 1 --2- 172.21.15.60:0/3706359203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04f012f610 0x7f04f009fca0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:04.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.062+0000 7f04f8430640 1 -- 172.21.15.60:0/3706359203 shutdown_connections 2023-08-24T10:07:04.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.062+0000 7f04f8430640 1 --2- 172.21.15.60:0/3706359203 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f04d4037130 0x7f04d4039640 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:04.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.062+0000 7f04f8430640 1 --2- 172.21.15.60:0/3706359203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04f012f610 0x7f04f009fca0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:04.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.062+0000 7f04f8430640 1 -- 172.21.15.60:0/3706359203 >> 172.21.15.60:0/3706359203 conn(0x7f04f00a2500 msgr2=0x7f04f01ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:04.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.062+0000 7f04f8430640 1 -- 172.21.15.60:0/3706359203 shutdown_connections 2023-08-24T10:07:04.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.062+0000 7f04f8430640 1 -- 172.21.15.60:0/3706359203 wait complete. 2023-08-24T10:07:04.073 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:07:04.073 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:04.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.180+0000 7fd4e2c45640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:04.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.184+0000 7fd4e2c45640 1 Processor -- start 2023-08-24T10:07:04.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.184+0000 7fd4e2c45640 1 -- start start 2023-08-24T10:07:04.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.184+0000 7fd4e2c45640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4dc06d300 0x7fd4dc12bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:04.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.184+0000 7fd4e2c45640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4dc099440 con 0x7fd4dc12c1c0 2023-08-24T10:07:04.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.184+0000 7fd4e2c45640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4dc09a050 con 0x7fd4dc06d300 2023-08-24T10:07:04.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.185+0000 7fd4e09e1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4dc06d300 0x7fd4dc12bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:04.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.185+0000 7fd4e09e1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4dc06d300 0x7fd4dc12bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43228/0 (socket says 172.21.15.60:43228) 2023-08-24T10:07:04.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.185+0000 7fd4dbfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd4dc12c1c0 0x7fd4dc12ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45356/0 (socket says 172.21.15.60:45356) 2023-08-24T10:07:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.185+0000 7fd4e09e1640 1 -- 172.21.15.60:0/2143730247 learned_addr learned my addr 172.21.15.60:0/2143730247 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.185+0000 7fd4e09e1640 1 -- 172.21.15.60:0/2143730247 >> v1:172.21.15.60:6789/0 conn(0x7fd4dc12c1c0 legacy=0x7fd4dc12ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.185+0000 7fd4e09e1640 1 -- 172.21.15.60:0/2143730247 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc12b9d0 con 0x7fd4dc06d300 2023-08-24T10:07:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.185+0000 7fd4e09e1640 1 --2- 172.21.15.60:0/2143730247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4dc06d300 0x7fd4dc12bc80 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edb620bce98356d server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:04.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.185+0000 7fd4db7fe640 1 -- 172.21.15.60:0/2143730247 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd4d000ad60 con 0x7fd4dc06d300 2023-08-24T10:07:04.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.185+0000 7fd4db7fe640 1 -- 172.21.15.60:0/2143730247 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd4d000f340 con 0x7fd4dc06d300 2023-08-24T10:07:04.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.185+0000 7fd4db7fe640 1 -- 172.21.15.60:0/2143730247 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd4d000f5e0 con 0x7fd4dc06d300 2023-08-24T10:07:04.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.186+0000 7fd4e2c45640 1 -- 172.21.15.60:0/2143730247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4dc06d300 msgr2=0x7fd4dc12bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:04.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.186+0000 7fd4e2c45640 1 --2- 172.21.15.60:0/2143730247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4dc06d300 0x7fd4dc12bc80 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:04.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.186+0000 7fd4e2c45640 1 -- 172.21.15.60:0/2143730247 shutdown_connections 2023-08-24T10:07:04.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.186+0000 7fd4e2c45640 1 --2- 172.21.15.60:0/2143730247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4dc06d300 0x7fd4dc12bc80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:04.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.186+0000 7fd4e2c45640 1 -- 172.21.15.60:0/2143730247 >> 172.21.15.60:0/2143730247 conn(0x7fd4dc09e110 msgr2=0x7fd4dc09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:04.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.186+0000 7fd4e2c45640 1 -- 172.21.15.60:0/2143730247 shutdown_connections 2023-08-24T10:07:04.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.186+0000 7fd4e2c45640 1 -- 172.21.15.60:0/2143730247 wait complete. 2023-08-24T10:07:04.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.186+0000 7fd4e2c45640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:04.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.187+0000 7fd4e2c45640 1 Processor -- start 2023-08-24T10:07:04.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.187+0000 7fd4e2c45640 1 -- start start 2023-08-24T10:07:04.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.188+0000 7fd4e2c45640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4dc06d300 0x7fd4dc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:04.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.188+0000 7fd4e2c45640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc089370 con 0x7fd4dc06d300 2023-08-24T10:07:04.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.188+0000 7fd4e09e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4dc06d300 0x7fd4dc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:04.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.188+0000 7fd4e09e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4dc06d300 0x7fd4dc06e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43230/0 (socket says 172.21.15.60:43230) 2023-08-24T10:07:04.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.188+0000 7fd4e09e1640 1 -- 172.21.15.60:0/1549895868 learned_addr learned my addr 172.21.15.60:0/1549895868 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:04.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.188+0000 7fd4e09e1640 1 -- 172.21.15.60:0/1549895868 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc096f10 con 0x7fd4dc06d300 2023-08-24T10:07:04.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.188+0000 7fd4e09e1640 1 --2- 172.21.15.60:0/1549895868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4dc06d300 0x7fd4dc06e9f0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:04.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.188+0000 7fd4d97fa640 1 -- 172.21.15.60:0/1549895868 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd4d0018040 con 0x7fd4dc06d300 2023-08-24T10:07:04.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.189+0000 7fd4d97fa640 1 -- 172.21.15.60:0/1549895868 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd4d000fdf0 con 0x7fd4dc06d300 2023-08-24T10:07:04.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.189+0000 7fd4d97fa640 1 -- 172.21.15.60:0/1549895868 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd4d0013760 con 0x7fd4dc06d300 2023-08-24T10:07:04.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.189+0000 7fd4e2c45640 1 -- 172.21.15.60:0/1549895868 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4dc089560 con 0x7fd4dc06d300 2023-08-24T10:07:04.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.189+0000 7fd4e2c45640 1 -- 172.21.15.60:0/1549895868 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4dc088ff0 con 0x7fd4dc06d300 2023-08-24T10:07:04.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.189+0000 7fd4d97fa640 1 -- 172.21.15.60:0/1549895868 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fd4d001b040 con 0x7fd4dc06d300 2023-08-24T10:07:04.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.189+0000 7fd4e2c45640 1 -- 172.21.15.60:0/1549895868 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4dc06e840 con 0x7fd4dc06d300 2023-08-24T10:07:04.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.190+0000 7fd4d97fa640 1 --2- 172.21.15.60:0/1549895868 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd4c0037130 0x7fd4c0039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:04.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.190+0000 7fd4d97fa640 1 -- 172.21.15.60:0/1549895868 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fd4d0049450 con 0x7fd4dc06d300 2023-08-24T10:07:04.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.190+0000 7fd4dbfff640 1 --2- 172.21.15.60:0/1549895868 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd4c0037130 0x7fd4c0039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:04.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.190+0000 7fd4dbfff640 1 --2- 172.21.15.60:0/1549895868 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd4c0037130 0x7fd4c0039640 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:04.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.194+0000 7fd4d97fa640 1 -- 172.21.15.60:0/1549895868 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fd4d000ddc0 con 0x7fd4dc06d300 2023-08-24T10:07:04.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.366+0000 7fd4e2c45640 1 -- 172.21.15.60:0/1549895868 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd4dc06da20 con 0x7fd4c0037130 2023-08-24T10:07:04.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.367+0000 7fd4d97fa640 1 -- 172.21.15.60:0/1549895868 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7fd4c8007e80 con 0x7fd4c0037130 2023-08-24T10:07:04.369 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:04.369 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:04.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.369+0000 7fd4e2c45640 1 -- 172.21.15.60:0/1549895868 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd4c0037130 msgr2=0x7fd4c0039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:04.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.369+0000 7fd4e2c45640 1 --2- 172.21.15.60:0/1549895868 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd4c0037130 0x7fd4c0039640 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:04.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.369+0000 7fd4e2c45640 1 -- 172.21.15.60:0/1549895868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4dc06d300 msgr2=0x7fd4dc06e9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:04.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.369+0000 7fd4e2c45640 1 --2- 172.21.15.60:0/1549895868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4dc06d300 0x7fd4dc06e9f0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:04.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.369+0000 7fd4e2c45640 1 -- 172.21.15.60:0/1549895868 shutdown_connections 2023-08-24T10:07:04.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.369+0000 7fd4e2c45640 1 --2- 172.21.15.60:0/1549895868 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd4c0037130 0x7fd4c0039640 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:04.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.369+0000 7fd4e2c45640 1 --2- 172.21.15.60:0/1549895868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4dc06d300 0x7fd4dc06e9f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:04.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.369+0000 7fd4e2c45640 1 -- 172.21.15.60:0/1549895868 >> 172.21.15.60:0/1549895868 conn(0x7fd4dc09e110 msgr2=0x7fd4dc09bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:04.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.369+0000 7fd4e2c45640 1 -- 172.21.15.60:0/1549895868 shutdown_connections 2023-08-24T10:07:04.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:04.369+0000 7fd4e2c45640 1 -- 172.21.15.60:0/1549895868 wait complete. 2023-08-24T10:07:04.381 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2023-08-24T10:07:03.519856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:07.136 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~120s 2023-08-24T10:07:07.136 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~120s 2023-08-24T10:07:07.136 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~120s 2023-08-24T10:07:07.136 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~120s 2023-08-24T10:07:07.383 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:07.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.503+0000 7efc8acac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:07.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.508+0000 7efc8acac640 1 Processor -- start 2023-08-24T10:07:07.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.508+0000 7efc8acac640 1 -- start start 2023-08-24T10:07:07.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.508+0000 7efc8acac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efc84132b50 0x7efc84132f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:07.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.508+0000 7efc8acac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc84132610 con 0x7efc8412f110 2023-08-24T10:07:07.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.508+0000 7efc8acac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc8409c260 con 0x7efc84132b50 2023-08-24T10:07:07.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.508+0000 7efc88a48640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efc84132b50 0x7efc84132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:07.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.508+0000 7efc88a48640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efc84132b50 0x7efc84132f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41270/0 (socket says 172.21.15.60:41270) 2023-08-24T10:07:07.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.509+0000 7efc88a48640 1 -- 172.21.15.60:0/2897343511 learned_addr learned my addr 172.21.15.60:0/2897343511 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:07.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.509+0000 7efc88a48640 1 -- 172.21.15.60:0/2897343511 >> v1:172.21.15.60:6789/0 conn(0x7efc8412f110 legacy=0x7efc8412f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:07.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.509+0000 7efc88a48640 1 -- 172.21.15.60:0/2897343511 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc84132840 con 0x7efc84132b50 2023-08-24T10:07:07.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.509+0000 7efc88a48640 1 --2- 172.21.15.60:0/2897343511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc84132b50 0x7efc84132f60 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29053a6d88699987 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:07.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.509+0000 7efc837fe640 1 -- 172.21.15.60:0/2897343511 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efc78009dd0 con 0x7efc84132b50 2023-08-24T10:07:07.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.509+0000 7efc837fe640 1 -- 172.21.15.60:0/2897343511 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7efc7800e230 con 0x7efc84132b50 2023-08-24T10:07:07.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.509+0000 7efc837fe640 1 -- 172.21.15.60:0/2897343511 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efc7800e4d0 con 0x7efc84132b50 2023-08-24T10:07:07.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.510+0000 7efc8acac640 1 -- 172.21.15.60:0/2897343511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc84132b50 msgr2=0x7efc84132f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:07.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.510+0000 7efc8acac640 1 --2- 172.21.15.60:0/2897343511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc84132b50 0x7efc84132f60 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:07.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.510+0000 7efc8acac640 1 -- 172.21.15.60:0/2897343511 shutdown_connections 2023-08-24T10:07:07.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.510+0000 7efc8acac640 1 --2- 172.21.15.60:0/2897343511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc84132b50 0x7efc84132f60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:07.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.510+0000 7efc8acac640 1 -- 172.21.15.60:0/2897343511 >> 172.21.15.60:0/2897343511 conn(0x7efc840a26a0 msgr2=0x7efc840a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:07.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.510+0000 7efc8acac640 1 -- 172.21.15.60:0/2897343511 shutdown_connections 2023-08-24T10:07:07.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.510+0000 7efc8acac640 1 -- 172.21.15.60:0/2897343511 wait complete. 2023-08-24T10:07:07.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.510+0000 7efc8acac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:07.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.511+0000 7efc8acac640 1 Processor -- start 2023-08-24T10:07:07.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.511+0000 7efc8acac640 1 -- start start 2023-08-24T10:07:07.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.512+0000 7efc8acac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8412f110 0x7efc8409fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:07.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.512+0000 7efc8acac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8412cc80 con 0x7efc8412f110 2023-08-24T10:07:07.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.512+0000 7efc88a48640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8412f110 0x7efc8409fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:07.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.512+0000 7efc88a48640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8412f110 0x7efc8409fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41274/0 (socket says 172.21.15.60:41274) 2023-08-24T10:07:07.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.512+0000 7efc88a48640 1 -- 172.21.15.60:0/1776366807 learned_addr learned my addr 172.21.15.60:0/1776366807 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:07.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.512+0000 7efc88a48640 1 -- 172.21.15.60:0/1776366807 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8409f400 con 0x7efc8412f110 2023-08-24T10:07:07.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.512+0000 7efc88a48640 1 --2- 172.21.15.60:0/1776366807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8412f110 0x7efc8409fcd0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:07.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.513+0000 7efc817fa640 1 -- 172.21.15.60:0/1776366807 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efc78014410 con 0x7efc8412f110 2023-08-24T10:07:07.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.513+0000 7efc817fa640 1 -- 172.21.15.60:0/1776366807 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7efc78014570 con 0x7efc8412f110 2023-08-24T10:07:07.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.513+0000 7efc8acac640 1 -- 172.21.15.60:0/1776366807 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc840a18e0 con 0x7efc8412f110 2023-08-24T10:07:07.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.513+0000 7efc817fa640 1 -- 172.21.15.60:0/1776366807 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efc78014810 con 0x7efc8412f110 2023-08-24T10:07:07.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.513+0000 7efc8acac640 1 -- 172.21.15.60:0/1776366807 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc8406a1d0 con 0x7efc8412f110 2023-08-24T10:07:07.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.514+0000 7efc817fa640 1 -- 172.21.15.60:0/1776366807 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7efc780149c0 con 0x7efc8412f110 2023-08-24T10:07:07.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.514+0000 7efc8acac640 1 -- 172.21.15.60:0/1776366807 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc8411c2b0 con 0x7efc8412f110 2023-08-24T10:07:07.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.514+0000 7efc817fa640 1 --2- 172.21.15.60:0/1776366807 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efc68037130 0x7efc68039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:07.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.514+0000 7efc817fa640 1 -- 172.21.15.60:0/1776366807 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7efc78019040 con 0x7efc8412f110 2023-08-24T10:07:07.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.514+0000 7efc83fff640 1 --2- 172.21.15.60:0/1776366807 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efc68037130 0x7efc68039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:07.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.515+0000 7efc83fff640 1 --2- 172.21.15.60:0/1776366807 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efc68037130 0x7efc68039640 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:07.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.518+0000 7efc817fa640 1 -- 172.21.15.60:0/1776366807 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7efc7800f8f0 con 0x7efc8412f110 2023-08-24T10:07:07.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.690+0000 7efc8acac640 1 -- 172.21.15.60:0/1776366807 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc841334a0 con 0x7efc68037130 2023-08-24T10:07:07.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.691+0000 7efc817fa640 1 -- 172.21.15.60:0/1776366807 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7efc70001650 con 0x7efc68037130 2023-08-24T10:07:07.693 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:07.693 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:07.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.693+0000 7efc8acac640 1 -- 172.21.15.60:0/1776366807 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efc68037130 msgr2=0x7efc68039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.693+0000 7efc8acac640 1 --2- 172.21.15.60:0/1776366807 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efc68037130 0x7efc68039640 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.693+0000 7efc8acac640 1 -- 172.21.15.60:0/1776366807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8412f110 msgr2=0x7efc8409fcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.693+0000 7efc8acac640 1 --2- 172.21.15.60:0/1776366807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8412f110 0x7efc8409fcd0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.693+0000 7efc8acac640 1 -- 172.21.15.60:0/1776366807 shutdown_connections 2023-08-24T10:07:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.693+0000 7efc8acac640 1 --2- 172.21.15.60:0/1776366807 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efc68037130 0x7efc68039640 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.693+0000 7efc8acac640 1 --2- 172.21.15.60:0/1776366807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8412f110 0x7efc8409fcd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:07.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.693+0000 7efc8acac640 1 -- 172.21.15.60:0/1776366807 >> 172.21.15.60:0/1776366807 conn(0x7efc840a26a0 msgr2=0x7efc841ba920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:07.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.693+0000 7efc8acac640 1 -- 172.21.15.60:0/1776366807 shutdown_connections 2023-08-24T10:07:07.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.693+0000 7efc8acac640 1 -- 172.21.15.60:0/1776366807 wait complete. 2023-08-24T10:07:07.705 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2023-08-24T10:07:07.520620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:07.706 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:07.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.812+0000 7f39450ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:07.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.816+0000 7f39450ec640 1 Processor -- start 2023-08-24T10:07:07.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.817+0000 7f39450ec640 1 -- start start 2023-08-24T10:07:07.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.817+0000 7f39450ec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f394006d300 0x7f394012bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:07.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.817+0000 7f39450ec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3940099440 con 0x7f394012c1c0 2023-08-24T10:07:07.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.817+0000 7f39450ec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394009a050 con 0x7f394006d300 2023-08-24T10:07:07.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.817+0000 7f393ed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f394006d300 0x7f394012bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:07.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.817+0000 7f393ed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f394006d300 0x7f394012bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41278/0 (socket says 172.21.15.60:41278) 2023-08-24T10:07:07.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.817+0000 7f3937fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f394012c1c0 0x7f394012ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34906/0 (socket says 172.21.15.60:34906) 2023-08-24T10:07:07.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.817+0000 7f393ed9d640 1 -- 172.21.15.60:0/1775264060 learned_addr learned my addr 172.21.15.60:0/1775264060 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:07.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.818+0000 7f393ed9d640 1 -- 172.21.15.60:0/1775264060 >> v1:172.21.15.60:6789/0 conn(0x7f394012c1c0 legacy=0x7f394012ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:07.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.818+0000 7f393ed9d640 1 -- 172.21.15.60:0/1775264060 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394012b9d0 con 0x7f394006d300 2023-08-24T10:07:07.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.818+0000 7f393ed9d640 1 --2- 172.21.15.60:0/1775264060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394006d300 0x7f394012bc80 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3151dfb2a723b6fc server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:07.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.818+0000 7f393e59c640 1 -- 172.21.15.60:0/1775264060 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f393000ad60 con 0x7f394006d300 2023-08-24T10:07:07.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.818+0000 7f393e59c640 1 -- 172.21.15.60:0/1775264060 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f393000f340 con 0x7f394006d300 2023-08-24T10:07:07.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.818+0000 7f393e59c640 1 -- 172.21.15.60:0/1775264060 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f393000f5e0 con 0x7f394006d300 2023-08-24T10:07:07.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.819+0000 7f39450ec640 1 -- 172.21.15.60:0/1775264060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394006d300 msgr2=0x7f394012bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:07.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.819+0000 7f39450ec640 1 --2- 172.21.15.60:0/1775264060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394006d300 0x7f394012bc80 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:07.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.819+0000 7f39450ec640 1 -- 172.21.15.60:0/1775264060 shutdown_connections 2023-08-24T10:07:07.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.819+0000 7f39450ec640 1 --2- 172.21.15.60:0/1775264060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394006d300 0x7f394012bc80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:07.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.819+0000 7f39450ec640 1 -- 172.21.15.60:0/1775264060 >> 172.21.15.60:0/1775264060 conn(0x7f394009e110 msgr2=0x7f394009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:07.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.819+0000 7f39450ec640 1 -- 172.21.15.60:0/1775264060 shutdown_connections 2023-08-24T10:07:07.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.819+0000 7f39450ec640 1 -- 172.21.15.60:0/1775264060 wait complete. 2023-08-24T10:07:07.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.819+0000 7f39450ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:07.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.820+0000 7f39450ec640 1 Processor -- start 2023-08-24T10:07:07.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.820+0000 7f39450ec640 1 -- start start 2023-08-24T10:07:07.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.821+0000 7f39450ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394006d300 0x7f3940070c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:07.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.821+0000 7f39450ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394008a080 con 0x7f394006d300 2023-08-24T10:07:07.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.821+0000 7f393ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394006d300 0x7f3940070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:07.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.821+0000 7f393ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394006d300 0x7f3940070c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41284/0 (socket says 172.21.15.60:41284) 2023-08-24T10:07:07.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.821+0000 7f393ed9d640 1 -- 172.21.15.60:0/1034033947 learned_addr learned my addr 172.21.15.60:0/1034033947 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:07.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.821+0000 7f393ed9d640 1 -- 172.21.15.60:0/1034033947 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3940089280 con 0x7f394006d300 2023-08-24T10:07:07.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.821+0000 7f393ed9d640 1 --2- 172.21.15.60:0/1034033947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394006d300 0x7f3940070c00 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:07.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.821+0000 7f39377fe640 1 -- 172.21.15.60:0/1034033947 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f393000fd70 con 0x7f394006d300 2023-08-24T10:07:07.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.821+0000 7f39377fe640 1 -- 172.21.15.60:0/1034033947 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f393000fed0 con 0x7f394006d300 2023-08-24T10:07:07.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.821+0000 7f39377fe640 1 -- 172.21.15.60:0/1034033947 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3930011550 con 0x7f394006d300 2023-08-24T10:07:07.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.821+0000 7f39450ec640 1 -- 172.21.15.60:0/1034033947 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39400708f0 con 0x7f394006d300 2023-08-24T10:07:07.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.821+0000 7f39450ec640 1 -- 172.21.15.60:0/1034033947 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394008b3b0 con 0x7f394006d300 2023-08-24T10:07:07.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.822+0000 7f39377fe640 1 -- 172.21.15.60:0/1034033947 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f3930011700 con 0x7f394006d300 2023-08-24T10:07:07.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.822+0000 7f39450ec640 1 -- 172.21.15.60:0/1034033947 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394006d9f0 con 0x7f394006d300 2023-08-24T10:07:07.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.823+0000 7f39377fe640 1 --2- 172.21.15.60:0/1034033947 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3920037130 0x7f3920039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:07.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.823+0000 7f39377fe640 1 -- 172.21.15.60:0/1034033947 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f3930017040 con 0x7f394006d300 2023-08-24T10:07:07.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.823+0000 7f3937fff640 1 --2- 172.21.15.60:0/1034033947 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3920037130 0x7f3920039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:07.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.823+0000 7f3937fff640 1 --2- 172.21.15.60:0/1034033947 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3920037130 0x7f3920039640 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:07.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.827+0000 7f39377fe640 1 -- 172.21.15.60:0/1034033947 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f39300103b0 con 0x7f394006d300 2023-08-24T10:07:08.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:07.999+0000 7f39450ec640 1 -- 172.21.15.60:0/1034033947 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f394008b910 con 0x7f3920037130 2023-08-24T10:07:08.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.000+0000 7f39377fe640 1 -- 172.21.15.60:0/1034033947 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f394008b910 con 0x7f3920037130 2023-08-24T10:07:08.002 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:08.002 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:08.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.002+0000 7f39450ec640 1 -- 172.21.15.60:0/1034033947 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3920037130 msgr2=0x7f3920039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:08.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.002+0000 7f39450ec640 1 --2- 172.21.15.60:0/1034033947 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3920037130 0x7f3920039640 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:08.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.002+0000 7f39450ec640 1 -- 172.21.15.60:0/1034033947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394006d300 msgr2=0x7f3940070c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:08.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.002+0000 7f39450ec640 1 --2- 172.21.15.60:0/1034033947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394006d300 0x7f3940070c00 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:08.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.002+0000 7f39450ec640 1 -- 172.21.15.60:0/1034033947 shutdown_connections 2023-08-24T10:07:08.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.002+0000 7f39450ec640 1 --2- 172.21.15.60:0/1034033947 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3920037130 0x7f3920039640 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:08.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.002+0000 7f39450ec640 1 --2- 172.21.15.60:0/1034033947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f394006d300 0x7f3940070c00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:08.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.002+0000 7f39450ec640 1 -- 172.21.15.60:0/1034033947 >> 172.21.15.60:0/1034033947 conn(0x7f394009e110 msgr2=0x7f394009bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:08.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.002+0000 7f39450ec640 1 -- 172.21.15.60:0/1034033947 shutdown_connections 2023-08-24T10:07:08.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.002+0000 7f39450ec640 1 -- 172.21.15.60:0/1034033947 wait complete. 2023-08-24T10:07:08.014 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2023-08-24T10:07:07.520620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:08.016 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:07:08.016 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:08.017 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:07:08.017 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:08.017 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:07:08.018 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:08.018 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:07:08.019 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:08.020 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:07:08.020 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:08.021 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:07:08.021 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:08.022 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:07:08.022 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:08.023 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:07:08.023 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:08.024 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:07:08.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:08.025 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:07:08.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.132+0000 7f98cc75e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:08.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.135+0000 7f98cc75e640 1 Processor -- start 2023-08-24T10:07:08.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.135+0000 7f98cc75e640 1 -- start start 2023-08-24T10:07:08.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.136+0000 7f98cc75e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98c4131d20 0x7f98c4134140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:08.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.136+0000 7f98cc75e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c409a650 con 0x7f98c412ece0 2023-08-24T10:07:08.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.136+0000 7f98cc75e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c406da00 con 0x7f98c4131d20 2023-08-24T10:07:08.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.136+0000 7f98ca4fa640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f98c412ece0 0x7f98c412f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34908/0 (socket says 172.21.15.60:34908) 2023-08-24T10:07:08.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.136+0000 7f98ca4fa640 1 -- 172.21.15.60:0/2457127397 learned_addr learned my addr 172.21.15.60:0/2457127397 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:07:08.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.136+0000 7f98c9cf9640 1 --2- 172.21.15.60:0/2457127397 >> v2:172.21.15.60:3300/0 conn(0x7f98c4131d20 0x7f98c4134140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:08.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.136+0000 7f98c9cf9640 1 -- 172.21.15.60:0/2457127397 >> v1:172.21.15.60:6789/0 conn(0x7f98c412ece0 legacy=0x7f98c412f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:08.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.136+0000 7f98c9cf9640 1 -- 172.21.15.60:0/2457127397 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c412ea10 con 0x7f98c4131d20 2023-08-24T10:07:08.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.136+0000 7f98c9cf9640 1 --2- 172.21.15.60:0/2457127397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98c4131d20 0x7f98c4134140 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff5d4f9490c0187 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:08.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.136+0000 7f98c94f8640 1 -- 172.21.15.60:0/2457127397 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f98b4009ce0 con 0x7f98c4131d20 2023-08-24T10:07:08.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.136+0000 7f98c94f8640 1 -- 172.21.15.60:0/2457127397 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f98b4009e40 con 0x7f98c4131d20 2023-08-24T10:07:08.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.136+0000 7f98c94f8640 1 -- 172.21.15.60:0/2457127397 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f98b400f470 con 0x7f98c4131d20 2023-08-24T10:07:08.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.137+0000 7f98cc75e640 1 -- 172.21.15.60:0/2457127397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98c4131d20 msgr2=0x7f98c4134140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:08.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.137+0000 7f98cc75e640 1 --2- 172.21.15.60:0/2457127397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98c4131d20 0x7f98c4134140 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:08.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.137+0000 7f98cc75e640 1 -- 172.21.15.60:0/2457127397 shutdown_connections 2023-08-24T10:07:08.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.137+0000 7f98cc75e640 1 --2- 172.21.15.60:0/2457127397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98c4131d20 0x7f98c4134140 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:08.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.137+0000 7f98cc75e640 1 -- 172.21.15.60:0/2457127397 >> 172.21.15.60:0/2457127397 conn(0x7f98c409e110 msgr2=0x7f98c409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:08.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.137+0000 7f98cc75e640 1 -- 172.21.15.60:0/2457127397 shutdown_connections 2023-08-24T10:07:08.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.137+0000 7f98cc75e640 1 -- 172.21.15.60:0/2457127397 wait complete. 2023-08-24T10:07:08.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.137+0000 7f98cc75e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:08.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.138+0000 7f98cc75e640 1 Processor -- start 2023-08-24T10:07:08.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.138+0000 7f98cc75e640 1 -- start start 2023-08-24T10:07:08.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.139+0000 7f98cc75e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98c412ece0 0x7f98c4070060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:08.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.139+0000 7f98cc75e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c4071450 con 0x7f98c412ece0 2023-08-24T10:07:08.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.139+0000 7f98ca4fa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98c412ece0 0x7f98c4070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:08.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.139+0000 7f98ca4fa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98c412ece0 0x7f98c4070060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41300/0 (socket says 172.21.15.60:41300) 2023-08-24T10:07:08.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.139+0000 7f98ca4fa640 1 -- 172.21.15.60:0/1945520992 learned_addr learned my addr 172.21.15.60:0/1945520992 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:08.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.139+0000 7f98ca4fa640 1 -- 172.21.15.60:0/1945520992 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c4129610 con 0x7f98c412ece0 2023-08-24T10:07:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.139+0000 7f98ca4fa640 1 --2- 172.21.15.60:0/1945520992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98c412ece0 0x7f98c4070060 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.140+0000 7f98baffd640 1 -- 172.21.15.60:0/1945520992 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f98c00059d0 con 0x7f98c412ece0 2023-08-24T10:07:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.140+0000 7f98baffd640 1 -- 172.21.15.60:0/1945520992 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f98c0005b30 con 0x7f98c412ece0 2023-08-24T10:07:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.140+0000 7f98cc75e640 1 -- 172.21.15.60:0/1945520992 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c4128a20 con 0x7f98c412ece0 2023-08-24T10:07:08.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.140+0000 7f98baffd640 1 -- 172.21.15.60:0/1945520992 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f98c0005dd0 con 0x7f98c412ece0 2023-08-24T10:07:08.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.140+0000 7f98cc75e640 1 -- 172.21.15.60:0/1945520992 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c4127990 con 0x7f98c412ece0 2023-08-24T10:07:08.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.141+0000 7f98baffd640 1 -- 172.21.15.60:0/1945520992 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f98c00070b0 con 0x7f98c412ece0 2023-08-24T10:07:08.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.141+0000 7f98cc75e640 1 -- 172.21.15.60:0/1945520992 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c4061800 con 0x7f98c412ece0 2023-08-24T10:07:08.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.141+0000 7f98baffd640 1 --2- 172.21.15.60:0/1945520992 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f98a8037130 0x7f98a8039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:08.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.141+0000 7f98baffd640 1 -- 172.21.15.60:0/1945520992 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f98c0046340 con 0x7f98c412ece0 2023-08-24T10:07:08.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.141+0000 7f98c9cf9640 1 --2- 172.21.15.60:0/1945520992 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f98a8037130 0x7f98a8039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:08.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.142+0000 7f98c9cf9640 1 --2- 172.21.15.60:0/1945520992 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f98a8037130 0x7f98a8039640 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:08.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.146+0000 7f98baffd640 1 -- 172.21.15.60:0/1945520992 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f98c000d330 con 0x7f98c412ece0 2023-08-24T10:07:08.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.389+0000 7f98cc75e640 1 -- 172.21.15.60:0/1945520992 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f98c4073730 con 0x7f98c412ece0 2023-08-24T10:07:08.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.390+0000 7f98baffd640 1 -- 172.21.15.60:0/1945520992 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f98c000d150 con 0x7f98c412ece0 2023-08-24T10:07:08.392 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:08.392 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":139,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:07:08.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.392+0000 7f98cc75e640 1 -- 172.21.15.60:0/1945520992 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f98a8037130 msgr2=0x7f98a8039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:08.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.392+0000 7f98cc75e640 1 --2- 172.21.15.60:0/1945520992 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f98a8037130 0x7f98a8039640 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:08.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.392+0000 7f98cc75e640 1 -- 172.21.15.60:0/1945520992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98c412ece0 msgr2=0x7f98c4070060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:08.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.392+0000 7f98cc75e640 1 --2- 172.21.15.60:0/1945520992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98c412ece0 0x7f98c4070060 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:08.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.392+0000 7f98cc75e640 1 -- 172.21.15.60:0/1945520992 shutdown_connections 2023-08-24T10:07:08.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.392+0000 7f98cc75e640 1 --2- 172.21.15.60:0/1945520992 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f98a8037130 0x7f98a8039640 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:08.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.392+0000 7f98cc75e640 1 --2- 172.21.15.60:0/1945520992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98c412ece0 0x7f98c4070060 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:08.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.392+0000 7f98cc75e640 1 -- 172.21.15.60:0/1945520992 >> 172.21.15.60:0/1945520992 conn(0x7f98c409e110 msgr2=0x7f98c4096820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:08.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.392+0000 7f98cc75e640 1 -- 172.21.15.60:0/1945520992 shutdown_connections 2023-08-24T10:07:08.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.392+0000 7f98cc75e640 1 -- 172.21.15.60:0/1945520992 wait complete. 2023-08-24T10:07:08.404 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:07:08.404 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:08.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.511+0000 7f67a0b1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:08.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.515+0000 7f67a0b1e640 1 Processor -- start 2023-08-24T10:07:08.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.515+0000 7f67a0b1e640 1 -- start start 2023-08-24T10:07:08.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.516+0000 7f67a0b1e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f679c132630 0x7f679c134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:08.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.516+0000 7f67a0b1e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f679c06e320 con 0x7f679c096860 2023-08-24T10:07:08.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.516+0000 7f67a0b1e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f679c06db70 con 0x7f679c132630 2023-08-24T10:07:08.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.516+0000 7f6793fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f679c132630 0x7f679c134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:08.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.516+0000 7f6793fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f679c132630 0x7f679c134a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41304/0 (socket says 172.21.15.60:41304) 2023-08-24T10:07:08.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.516+0000 7f6793fff640 1 -- 172.21.15.60:0/2898692000 learned_addr learned my addr 172.21.15.60:0/2898692000 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:08.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.516+0000 7f679a59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f679c096860 0x7f679c096c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34922/0 (socket says 172.21.15.60:34922) 2023-08-24T10:07:08.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.516+0000 7f6793fff640 1 -- 172.21.15.60:0/2898692000 >> v1:172.21.15.60:6789/0 conn(0x7f679c096860 legacy=0x7f679c096c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:08.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.516+0000 7f6793fff640 1 -- 172.21.15.60:0/2898692000 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c06cef0 con 0x7f679c132630 2023-08-24T10:07:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.516+0000 7f6793fff640 1 --2- 172.21.15.60:0/2898692000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c132630 0x7f679c134a50 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ecd4d63d60a737 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.516+0000 7f6799d9b640 1 -- 172.21.15.60:0/2898692000 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6784009dd0 con 0x7f679c132630 2023-08-24T10:07:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.516+0000 7f6799d9b640 1 -- 172.21.15.60:0/2898692000 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f678400e230 con 0x7f679c132630 2023-08-24T10:07:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.516+0000 7f6799d9b640 1 -- 172.21.15.60:0/2898692000 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f678400e4d0 con 0x7f679c132630 2023-08-24T10:07:08.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.517+0000 7f67a0b1e640 1 -- 172.21.15.60:0/2898692000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c132630 msgr2=0x7f679c134a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:08.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.517+0000 7f67a0b1e640 1 --2- 172.21.15.60:0/2898692000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c132630 0x7f679c134a50 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:08.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.517+0000 7f67a0b1e640 1 -- 172.21.15.60:0/2898692000 shutdown_connections 2023-08-24T10:07:08.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.517+0000 7f67a0b1e640 1 --2- 172.21.15.60:0/2898692000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c132630 0x7f679c134a50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:08.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.517+0000 7f67a0b1e640 1 -- 172.21.15.60:0/2898692000 >> 172.21.15.60:0/2898692000 conn(0x7f679c09e110 msgr2=0x7f679c09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:08.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.517+0000 7f67a0b1e640 1 -- 172.21.15.60:0/2898692000 shutdown_connections 2023-08-24T10:07:08.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.517+0000 7f67a0b1e640 1 -- 172.21.15.60:0/2898692000 wait complete. 2023-08-24T10:07:08.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.517+0000 7f67a0b1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:08.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.518+0000 7f67a0b1e640 1 Processor -- start 2023-08-24T10:07:08.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.518+0000 7f67a0b1e640 1 -- start start 2023-08-24T10:07:08.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.519+0000 7f67a0b1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c096860 0x7f679c1281a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:08.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.519+0000 7f67a0b1e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c12bd50 con 0x7f679c096860 2023-08-24T10:07:08.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.519+0000 7f679a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c096860 0x7f679c1281a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:08.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.519+0000 7f679a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c096860 0x7f679c1281a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41308/0 (socket says 172.21.15.60:41308) 2023-08-24T10:07:08.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.519+0000 7f679a59c640 1 -- 172.21.15.60:0/103319807 learned_addr learned my addr 172.21.15.60:0/103319807 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:08.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.519+0000 7f679a59c640 1 -- 172.21.15.60:0/103319807 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c127b60 con 0x7f679c096860 2023-08-24T10:07:08.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.520+0000 7f679a59c640 1 --2- 172.21.15.60:0/103319807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c096860 0x7f679c1281a0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:08.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.520+0000 7f6792ffd640 1 -- 172.21.15.60:0/103319807 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f678c0071e0 con 0x7f679c096860 2023-08-24T10:07:08.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.520+0000 7f6792ffd640 1 -- 172.21.15.60:0/103319807 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f678c007340 con 0x7f679c096860 2023-08-24T10:07:08.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.520+0000 7f67a0b1e640 1 -- 172.21.15.60:0/103319807 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679c127e00 con 0x7f679c096860 2023-08-24T10:07:08.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.520+0000 7f6792ffd640 1 -- 172.21.15.60:0/103319807 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f678c0075e0 con 0x7f679c096860 2023-08-24T10:07:08.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.520+0000 7f67a0b1e640 1 -- 172.21.15.60:0/103319807 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679c12b0f0 con 0x7f679c096860 2023-08-24T10:07:08.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.521+0000 7f6792ffd640 1 -- 172.21.15.60:0/103319807 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f678c0057e0 con 0x7f679c096860 2023-08-24T10:07:08.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.521+0000 7f67a0b1e640 1 -- 172.21.15.60:0/103319807 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f679c12a460 con 0x7f679c096860 2023-08-24T10:07:08.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.521+0000 7f6792ffd640 1 --2- 172.21.15.60:0/103319807 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f677c037130 0x7f677c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:08.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.521+0000 7f6792ffd640 1 -- 172.21.15.60:0/103319807 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f678c0463b0 con 0x7f679c096860 2023-08-24T10:07:08.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.521+0000 7f6793fff640 1 --2- 172.21.15.60:0/103319807 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f677c037130 0x7f677c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:08.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.522+0000 7f6793fff640 1 --2- 172.21.15.60:0/103319807 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f677c037130 0x7f677c039640 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:08.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.526+0000 7f6792ffd640 1 -- 172.21.15.60:0/103319807 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f678c00d330 con 0x7f679c096860 2023-08-24T10:07:08.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.699+0000 7f67a0b1e640 1 -- 172.21.15.60:0/103319807 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f679c06d190 con 0x7f677c037130 2023-08-24T10:07:08.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.700+0000 7f6792ffd640 1 -- 172.21.15.60:0/103319807 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f6784009df0 con 0x7f677c037130 2023-08-24T10:07:08.702 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:08.702 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:08.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.702+0000 7f67a0b1e640 1 -- 172.21.15.60:0/103319807 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f677c037130 msgr2=0x7f677c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:08.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.702+0000 7f67a0b1e640 1 --2- 172.21.15.60:0/103319807 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f677c037130 0x7f677c039640 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:08.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.702+0000 7f67a0b1e640 1 -- 172.21.15.60:0/103319807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c096860 msgr2=0x7f679c1281a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:08.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.702+0000 7f67a0b1e640 1 --2- 172.21.15.60:0/103319807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c096860 0x7f679c1281a0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:08.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.702+0000 7f67a0b1e640 1 -- 172.21.15.60:0/103319807 shutdown_connections 2023-08-24T10:07:08.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.702+0000 7f67a0b1e640 1 --2- 172.21.15.60:0/103319807 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f677c037130 0x7f677c039640 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:08.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.702+0000 7f67a0b1e640 1 --2- 172.21.15.60:0/103319807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c096860 0x7f679c1281a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:08.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.702+0000 7f67a0b1e640 1 -- 172.21.15.60:0/103319807 >> 172.21.15.60:0/103319807 conn(0x7f679c09e110 msgr2=0x7f679c0950c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:08.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.702+0000 7f67a0b1e640 1 -- 172.21.15.60:0/103319807 shutdown_connections 2023-08-24T10:07:08.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:08.702+0000 7f67a0b1e640 1 -- 172.21.15.60:0/103319807 wait complete. 2023-08-24T10:07:08.714 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2023-08-24T10:07:07.520620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:11.716 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:11.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.837+0000 7f7ebaeac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:11.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.841+0000 7f7ebaeac640 1 Processor -- start 2023-08-24T10:07:11.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.841+0000 7f7ebaeac640 1 -- start start 2023-08-24T10:07:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.842+0000 7f7ebaeac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7eb4099f90 0x7f7eb412c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.842+0000 7f7ebaeac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb4099b40 con 0x7f7eb412c800 2023-08-24T10:07:11.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.842+0000 7f7ebaeac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eb412be70 con 0x7f7eb4099f90 2023-08-24T10:07:11.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.842+0000 7f7eb8c48640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7eb4099f90 0x7f7eb412c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:11.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.842+0000 7f7eb8c48640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7eb4099f90 0x7f7eb412c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41324/0 (socket says 172.21.15.60:41324) 2023-08-24T10:07:11.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.842+0000 7f7eb8c48640 1 -- 172.21.15.60:0/3428824363 learned_addr learned my addr 172.21.15.60:0/3428824363 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:11.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.842+0000 7f7eabfff640 1 --1- 172.21.15.60:0/3428824363 >> v1:172.21.15.60:6789/0 conn(0x7f7eb412c800 0x7f7eb40986b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34932/0 (socket says 172.21.15.60:34932) 2023-08-24T10:07:11.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.842+0000 7f7eb8c48640 1 -- 172.21.15.60:0/3428824363 >> v1:172.21.15.60:6789/0 conn(0x7f7eb412c800 legacy=0x7f7eb40986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:11.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.842+0000 7f7eb8c48640 1 -- 172.21.15.60:0/3428824363 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb409b1b0 con 0x7f7eb4099f90 2023-08-24T10:07:11.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.842+0000 7f7eb8c48640 1 --2- 172.21.15.60:0/3428824363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7eb4099f90 0x7f7eb412c2c0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f869bc83edc1f7 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:11.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.842+0000 7f7eab7fe640 1 -- 172.21.15.60:0/3428824363 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7eb0009dd0 con 0x7f7eb4099f90 2023-08-24T10:07:11.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.842+0000 7f7eab7fe640 1 -- 172.21.15.60:0/3428824363 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7eb000e230 con 0x7f7eb4099f90 2023-08-24T10:07:11.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.842+0000 7f7eab7fe640 1 -- 172.21.15.60:0/3428824363 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7eb000e4d0 con 0x7f7eb4099f90 2023-08-24T10:07:11.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.843+0000 7f7ebaeac640 1 -- 172.21.15.60:0/3428824363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7eb4099f90 msgr2=0x7f7eb412c2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:11.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.843+0000 7f7ebaeac640 1 --2- 172.21.15.60:0/3428824363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7eb4099f90 0x7f7eb412c2c0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:11.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.843+0000 7f7ebaeac640 1 -- 172.21.15.60:0/3428824363 shutdown_connections 2023-08-24T10:07:11.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.843+0000 7f7ebaeac640 1 --2- 172.21.15.60:0/3428824363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7eb4099f90 0x7f7eb412c2c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:11.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.843+0000 7f7ebaeac640 1 -- 172.21.15.60:0/3428824363 >> 172.21.15.60:0/3428824363 conn(0x7f7eb409f290 msgr2=0x7f7eb409f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.843+0000 7f7ebaeac640 1 -- 172.21.15.60:0/3428824363 shutdown_connections 2023-08-24T10:07:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.843+0000 7f7ebaeac640 1 -- 172.21.15.60:0/3428824363 wait complete. 2023-08-24T10:07:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.843+0000 7f7ebaeac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:11.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.844+0000 7f7ebaeac640 1 Processor -- start 2023-08-24T10:07:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.844+0000 7f7ebaeac640 1 -- start start 2023-08-24T10:07:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.845+0000 7f7ebaeac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7eb4099f90 0x7f7eb4070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.845+0000 7f7ebaeac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb4072050 con 0x7f7eb4099f90 2023-08-24T10:07:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.845+0000 7f7eb8c48640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7eb4099f90 0x7f7eb4070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.845+0000 7f7eb8c48640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7eb4099f90 0x7f7eb4070340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41336/0 (socket says 172.21.15.60:41336) 2023-08-24T10:07:11.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.845+0000 7f7eb8c48640 1 -- 172.21.15.60:0/473981287 learned_addr learned my addr 172.21.15.60:0/473981287 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:11.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.846+0000 7f7eb8c48640 1 -- 172.21.15.60:0/473981287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb4070120 con 0x7f7eb4099f90 2023-08-24T10:07:11.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.846+0000 7f7eb8c48640 1 --2- 172.21.15.60:0/473981287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7eb4099f90 0x7f7eb4070340 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:11.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.846+0000 7f7ea97fa640 1 -- 172.21.15.60:0/473981287 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7eb0014540 con 0x7f7eb4099f90 2023-08-24T10:07:11.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.846+0000 7f7ea97fa640 1 -- 172.21.15.60:0/473981287 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7eb00146a0 con 0x7f7eb4099f90 2023-08-24T10:07:11.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.846+0000 7f7ea97fa640 1 -- 172.21.15.60:0/473981287 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7eb0014940 con 0x7f7eb4099f90 2023-08-24T10:07:11.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.846+0000 7f7ebaeac640 1 -- 172.21.15.60:0/473981287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb4117a30 con 0x7f7eb4099f90 2023-08-24T10:07:11.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.846+0000 7f7ebaeac640 1 -- 172.21.15.60:0/473981287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb406fd80 con 0x7f7eb4099f90 2023-08-24T10:07:11.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.847+0000 7f7ea97fa640 1 -- 172.21.15.60:0/473981287 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f7eb0014af0 con 0x7f7eb4099f90 2023-08-24T10:07:11.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.847+0000 7f7ebaeac640 1 -- 172.21.15.60:0/473981287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb4061800 con 0x7f7eb4099f90 2023-08-24T10:07:11.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.847+0000 7f7ea97fa640 1 --2- 172.21.15.60:0/473981287 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7ea0037130 0x7f7ea0039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:11.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.847+0000 7f7ea97fa640 1 -- 172.21.15.60:0/473981287 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f7eb0019040 con 0x7f7eb4099f90 2023-08-24T10:07:11.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.848+0000 7f7eabfff640 1 --2- 172.21.15.60:0/473981287 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7ea0037130 0x7f7ea0039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:11.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.848+0000 7f7eabfff640 1 --2- 172.21.15.60:0/473981287 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7ea0037130 0x7f7ea0039640 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:11.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:11.852+0000 7f7ea97fa640 1 -- 172.21.15.60:0/473981287 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f7eb0018190 con 0x7f7eb4099f90 2023-08-24T10:07:12.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.024+0000 7f7ebaeac640 1 -- 172.21.15.60:0/473981287 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7eb412cc20 con 0x7f7ea0037130 2023-08-24T10:07:12.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.025+0000 7f7ea97fa640 1 -- 172.21.15.60:0/473981287 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f7eb412cc20 con 0x7f7ea0037130 2023-08-24T10:07:12.026 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:12.027 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.027+0000 7f7ebaeac640 1 -- 172.21.15.60:0/473981287 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7ea0037130 msgr2=0x7f7ea0039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.027+0000 7f7ebaeac640 1 --2- 172.21.15.60:0/473981287 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7ea0037130 0x7f7ea0039640 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.027+0000 7f7ebaeac640 1 -- 172.21.15.60:0/473981287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7eb4099f90 msgr2=0x7f7eb4070340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.027+0000 7f7ebaeac640 1 --2- 172.21.15.60:0/473981287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7eb4099f90 0x7f7eb4070340 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.027+0000 7f7ebaeac640 1 -- 172.21.15.60:0/473981287 shutdown_connections 2023-08-24T10:07:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.027+0000 7f7ebaeac640 1 --2- 172.21.15.60:0/473981287 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7ea0037130 0x7f7ea0039640 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:12.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.027+0000 7f7ebaeac640 1 --2- 172.21.15.60:0/473981287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7eb4099f90 0x7f7eb4070340 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:12.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.027+0000 7f7ebaeac640 1 -- 172.21.15.60:0/473981287 >> 172.21.15.60:0/473981287 conn(0x7f7eb409f290 msgr2=0x7f7eb41b73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:12.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.027+0000 7f7ebaeac640 1 -- 172.21.15.60:0/473981287 shutdown_connections 2023-08-24T10:07:12.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.027+0000 7f7ebaeac640 1 -- 172.21.15.60:0/473981287 wait complete. 2023-08-24T10:07:12.038 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2023-08-24T10:07:11.521325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:12.039 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:12.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.146+0000 7fc60775e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:12.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.150+0000 7fc60775e640 1 Processor -- start 2023-08-24T10:07:12.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.150+0000 7fc60775e640 1 -- start start 2023-08-24T10:07:12.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.151+0000 7fc60775e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc600131d40 0x7fc600134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:12.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.151+0000 7fc60775e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc600099440 con 0x7fc60006d300 2023-08-24T10:07:12.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.151+0000 7fc60775e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc60009a050 con 0x7fc600131d40 2023-08-24T10:07:12.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.151+0000 7fc604cf9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc600131d40 0x7fc600134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:12.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.151+0000 7fc604cf9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc600131d40 0x7fc600134160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41342/0 (socket says 172.21.15.60:41342) 2023-08-24T10:07:12.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.151+0000 7fc604cf9640 1 -- 172.21.15.60:0/517800997 learned_addr learned my addr 172.21.15.60:0/517800997 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:12.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.151+0000 7fc604cf9640 1 -- 172.21.15.60:0/517800997 >> v1:172.21.15.60:6789/0 conn(0x7fc60006d300 legacy=0x7fc60012bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:12.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.151+0000 7fc604cf9640 1 -- 172.21.15.60:0/517800997 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60012b9d0 con 0x7fc600131d40 2023-08-24T10:07:12.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.151+0000 7fc604cf9640 1 --2- 172.21.15.60:0/517800997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc600131d40 0x7fc600134160 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9785771718bcade server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:12.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.151+0000 7fc5f7fff640 1 -- 172.21.15.60:0/517800997 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc5f0009dc0 con 0x7fc600131d40 2023-08-24T10:07:12.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.151+0000 7fc5f7fff640 1 -- 172.21.15.60:0/517800997 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc5f000e230 con 0x7fc600131d40 2023-08-24T10:07:12.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.151+0000 7fc5f7fff640 1 -- 172.21.15.60:0/517800997 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc5f000e4d0 con 0x7fc600131d40 2023-08-24T10:07:12.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.152+0000 7fc60775e640 1 -- 172.21.15.60:0/517800997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc600131d40 msgr2=0x7fc600134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:12.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.152+0000 7fc60775e640 1 --2- 172.21.15.60:0/517800997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc600131d40 0x7fc600134160 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:12.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.152+0000 7fc60775e640 1 -- 172.21.15.60:0/517800997 shutdown_connections 2023-08-24T10:07:12.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.152+0000 7fc60775e640 1 --2- 172.21.15.60:0/517800997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc600131d40 0x7fc600134160 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:12.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.152+0000 7fc60775e640 1 -- 172.21.15.60:0/517800997 >> 172.21.15.60:0/517800997 conn(0x7fc60009e110 msgr2=0x7fc60009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:12.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.152+0000 7fc60775e640 1 -- 172.21.15.60:0/517800997 shutdown_connections 2023-08-24T10:07:12.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.152+0000 7fc60775e640 1 -- 172.21.15.60:0/517800997 wait complete. 2023-08-24T10:07:12.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.152+0000 7fc60775e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:12.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.153+0000 7fc60775e640 1 Processor -- start 2023-08-24T10:07:12.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.153+0000 7fc60775e640 1 -- start start 2023-08-24T10:07:12.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.154+0000 7fc60775e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc60006d300 0x7fc600070130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:12.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.154+0000 7fc60775e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc600127630 con 0x7fc60006d300 2023-08-24T10:07:12.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.154+0000 7fc6054fa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc60006d300 0x7fc600070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:12.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.154+0000 7fc6054fa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc60006d300 0x7fc600070130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41348/0 (socket says 172.21.15.60:41348) 2023-08-24T10:07:12.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.154+0000 7fc6054fa640 1 -- 172.21.15.60:0/3230574550 learned_addr learned my addr 172.21.15.60:0/3230574550 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:12.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.154+0000 7fc6054fa640 1 -- 172.21.15.60:0/3230574550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc600126a20 con 0x7fc60006d300 2023-08-24T10:07:12.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.154+0000 7fc6054fa640 1 --2- 172.21.15.60:0/3230574550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc60006d300 0x7fc600070130 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:12.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.155+0000 7fc5f5ffb640 1 -- 172.21.15.60:0/3230574550 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc5fc00c860 con 0x7fc60006d300 2023-08-24T10:07:12.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.155+0000 7fc5f5ffb640 1 -- 172.21.15.60:0/3230574550 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc5fc00c9c0 con 0x7fc60006d300 2023-08-24T10:07:12.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.155+0000 7fc60775e640 1 -- 172.21.15.60:0/3230574550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60006b0e0 con 0x7fc60006d300 2023-08-24T10:07:12.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.155+0000 7fc5f5ffb640 1 -- 172.21.15.60:0/3230574550 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc5fc00cc60 con 0x7fc60006d300 2023-08-24T10:07:12.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.155+0000 7fc60775e640 1 -- 172.21.15.60:0/3230574550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc600125a00 con 0x7fc60006d300 2023-08-24T10:07:12.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.155+0000 7fc5f5ffb640 1 -- 172.21.15.60:0/3230574550 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fc5fc011310 con 0x7fc60006d300 2023-08-24T10:07:12.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.156+0000 7fc5f5ffb640 1 --2- 172.21.15.60:0/3230574550 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc5e40370e0 0x7fc5e40395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:12.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.156+0000 7fc5f5ffb640 1 -- 172.21.15.60:0/3230574550 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fc5fc046a70 con 0x7fc60006d300 2023-08-24T10:07:12.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.156+0000 7fc60775e640 1 -- 172.21.15.60:0/3230574550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc600126860 con 0x7fc60006d300 2023-08-24T10:07:12.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.156+0000 7fc604cf9640 1 --2- 172.21.15.60:0/3230574550 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc5e40370e0 0x7fc5e40395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:12.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.157+0000 7fc604cf9640 1 --2- 172.21.15.60:0/3230574550 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc5e40370e0 0x7fc5e40395f0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:12.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.161+0000 7fc5f5ffb640 1 -- 172.21.15.60:0/3230574550 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fc5fc00ce60 con 0x7fc60006d300 2023-08-24T10:07:12.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.335+0000 7fc60775e640 1 -- 172.21.15.60:0/3230574550 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc60012a650 con 0x7fc5e40370e0 2023-08-24T10:07:12.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.337+0000 7fc5f5ffb640 1 -- 172.21.15.60:0/3230574550 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7fc5f0009d90 con 0x7fc5e40370e0 2023-08-24T10:07:12.338 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:12.339 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:12.339 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~125s 2023-08-24T10:07:12.339 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~125s 2023-08-24T10:07:12.339 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~125s 2023-08-24T10:07:12.339 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~125s 2023-08-24T10:07:12.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.339+0000 7fc60775e640 1 -- 172.21.15.60:0/3230574550 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc5e40370e0 msgr2=0x7fc5e40395f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:12.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.339+0000 7fc60775e640 1 --2- 172.21.15.60:0/3230574550 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc5e40370e0 0x7fc5e40395f0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:12.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.339+0000 7fc60775e640 1 -- 172.21.15.60:0/3230574550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc60006d300 msgr2=0x7fc600070130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.339+0000 7fc60775e640 1 --2- 172.21.15.60:0/3230574550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc60006d300 0x7fc600070130 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.339+0000 7fc60775e640 1 -- 172.21.15.60:0/3230574550 shutdown_connections 2023-08-24T10:07:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.339+0000 7fc60775e640 1 --2- 172.21.15.60:0/3230574550 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc5e40370e0 0x7fc5e40395f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.339+0000 7fc60775e640 1 --2- 172.21.15.60:0/3230574550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc60006d300 0x7fc600070130 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.339+0000 7fc60775e640 1 -- 172.21.15.60:0/3230574550 >> 172.21.15.60:0/3230574550 conn(0x7fc60009e110 msgr2=0x7fc600096690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.339+0000 7fc60775e640 1 -- 172.21.15.60:0/3230574550 shutdown_connections 2023-08-24T10:07:12.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.339+0000 7fc60775e640 1 -- 172.21.15.60:0/3230574550 wait complete. 2023-08-24T10:07:12.350 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2023-08-24T10:07:11.521325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:12.351 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:07:12.352 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:12.352 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:07:12.352 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:12.353 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:07:12.353 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:12.353 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:07:12.353 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:12.354 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:07:12.354 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:12.354 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:07:12.354 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:12.355 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:07:12.355 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:12.356 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:07:12.356 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:12.356 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:07:12.357 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:12.357 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:07:12.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.464+0000 7f9602d9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:12.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.467+0000 7f9602d9e640 1 Processor -- start 2023-08-24T10:07:12.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.467+0000 7f9602d9e640 1 -- start start 2023-08-24T10:07:12.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.468+0000 7f9602d9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f95fc135a60 0x7f95fc137e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:12.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.468+0000 7f9602d9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95fc09d750 con 0x7f95fc12f610 2023-08-24T10:07:12.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.468+0000 7f9602d9e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95fc130130 con 0x7f95fc135a60 2023-08-24T10:07:12.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.468+0000 7f9600b3a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f95fc12f610 0x7f95fc12fa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34942/0 (socket says 172.21.15.60:34942) 2023-08-24T10:07:12.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.468+0000 7f9600b3a640 1 -- 172.21.15.60:0/3338962845 learned_addr learned my addr 172.21.15.60:0/3338962845 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:07:12.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.468+0000 7f95fbfff640 1 --2- 172.21.15.60:0/3338962845 >> v2:172.21.15.60:3300/0 conn(0x7f95fc135a60 0x7f95fc137e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:12.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.469+0000 7f95fbfff640 1 -- 172.21.15.60:0/3338962845 >> v1:172.21.15.60:6789/0 conn(0x7f95fc12f610 legacy=0x7f95fc12fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:12.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.469+0000 7f95fbfff640 1 -- 172.21.15.60:0/3338962845 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc09d980 con 0x7f95fc135a60 2023-08-24T10:07:12.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.469+0000 7f95fbfff640 1 --2- 172.21.15.60:0/3338962845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc135a60 0x7f95fc137e80 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc1c28fec94bb68 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:12.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.469+0000 7f95fb7fe640 1 -- 172.21.15.60:0/3338962845 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f95f0009cd0 con 0x7f95fc135a60 2023-08-24T10:07:12.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.469+0000 7f95fb7fe640 1 -- 172.21.15.60:0/3338962845 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f95f0009e30 con 0x7f95fc135a60 2023-08-24T10:07:12.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.469+0000 7f95fb7fe640 1 -- 172.21.15.60:0/3338962845 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f95f000e360 con 0x7f95fc135a60 2023-08-24T10:07:12.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.470+0000 7f9602d9e640 1 -- 172.21.15.60:0/3338962845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc135a60 msgr2=0x7f95fc137e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:12.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.470+0000 7f9602d9e640 1 --2- 172.21.15.60:0/3338962845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc135a60 0x7f95fc137e80 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:12.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.470+0000 7f9602d9e640 1 -- 172.21.15.60:0/3338962845 shutdown_connections 2023-08-24T10:07:12.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.470+0000 7f9602d9e640 1 --2- 172.21.15.60:0/3338962845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc135a60 0x7f95fc137e80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:12.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.470+0000 7f9602d9e640 1 -- 172.21.15.60:0/3338962845 >> 172.21.15.60:0/3338962845 conn(0x7f95fc0a2500 msgr2=0x7f95fc0a2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:12.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.470+0000 7f9602d9e640 1 -- 172.21.15.60:0/3338962845 shutdown_connections 2023-08-24T10:07:12.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.470+0000 7f9602d9e640 1 -- 172.21.15.60:0/3338962845 wait complete. 2023-08-24T10:07:12.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.470+0000 7f9602d9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:12.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.471+0000 7f9602d9e640 1 Processor -- start 2023-08-24T10:07:12.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.471+0000 7f9602d9e640 1 -- start start 2023-08-24T10:07:12.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.472+0000 7f9602d9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc12f610 0x7f95fc0a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:12.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.472+0000 7f9602d9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc0a08e0 con 0x7f95fc12f610 2023-08-24T10:07:12.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.472+0000 7f9600b3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc12f610 0x7f95fc0a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:12.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.472+0000 7f9600b3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc12f610 0x7f95fc0a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41360/0 (socket says 172.21.15.60:41360) 2023-08-24T10:07:12.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.472+0000 7f9600b3a640 1 -- 172.21.15.60:0/2108423596 learned_addr learned my addr 172.21.15.60:0/2108423596 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:12.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.472+0000 7f9600b3a640 1 -- 172.21.15.60:0/2108423596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc11d420 con 0x7f95fc12f610 2023-08-24T10:07:12.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.472+0000 7f9600b3a640 1 --2- 172.21.15.60:0/2108423596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc12f610 0x7f95fc0a0d10 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:12.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.472+0000 7f95f97fa640 1 -- 172.21.15.60:0/2108423596 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f95f40070b0 con 0x7f95fc12f610 2023-08-24T10:07:12.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.472+0000 7f95f97fa640 1 -- 172.21.15.60:0/2108423596 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f95f4007210 con 0x7f95fc12f610 2023-08-24T10:07:12.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.472+0000 7f9602d9e640 1 -- 172.21.15.60:0/2108423596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95fc0a0540 con 0x7f95fc12f610 2023-08-24T10:07:12.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.472+0000 7f95f97fa640 1 -- 172.21.15.60:0/2108423596 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f95f40074b0 con 0x7f95fc12f610 2023-08-24T10:07:12.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.472+0000 7f9602d9e640 1 -- 172.21.15.60:0/2108423596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95fc11c810 con 0x7f95fc12f610 2023-08-24T10:07:12.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.473+0000 7f95f97fa640 1 -- 172.21.15.60:0/2108423596 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f95f40076a0 con 0x7f95fc12f610 2023-08-24T10:07:12.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.473+0000 7f95f97fa640 1 --2- 172.21.15.60:0/2108423596 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f95e8037180 0x7f95e8039690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:12.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.473+0000 7f95f97fa640 1 -- 172.21.15.60:0/2108423596 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f95f4046f80 con 0x7f95fc12f610 2023-08-24T10:07:12.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.473+0000 7f9602d9e640 1 -- 172.21.15.60:0/2108423596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95fc061800 con 0x7f95fc12f610 2023-08-24T10:07:12.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.474+0000 7f95fbfff640 1 --2- 172.21.15.60:0/2108423596 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f95e8037180 0x7f95e8039690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:12.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.474+0000 7f95fbfff640 1 --2- 172.21.15.60:0/2108423596 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f95e8037180 0x7f95e8039690 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:12.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.478+0000 7f95f97fa640 1 -- 172.21.15.60:0/2108423596 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f95f40090b0 con 0x7f95fc12f610 2023-08-24T10:07:12.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.721+0000 7f9602d9e640 1 -- 172.21.15.60:0/2108423596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f95fc0a06d0 con 0x7f95fc12f610 2023-08-24T10:07:12.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.721+0000 7f95f97fa640 1 -- 172.21.15.60:0/2108423596 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f95f40092c0 con 0x7f95fc12f610 2023-08-24T10:07:12.723 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:12.723 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":143,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:07:12.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.723+0000 7f9602d9e640 1 -- 172.21.15.60:0/2108423596 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f95e8037180 msgr2=0x7f95e8039690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:12.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.723+0000 7f9602d9e640 1 --2- 172.21.15.60:0/2108423596 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f95e8037180 0x7f95e8039690 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:12.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.723+0000 7f9602d9e640 1 -- 172.21.15.60:0/2108423596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc12f610 msgr2=0x7f95fc0a0d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:12.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.723+0000 7f9602d9e640 1 --2- 172.21.15.60:0/2108423596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc12f610 0x7f95fc0a0d10 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:12.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.723+0000 7f9602d9e640 1 -- 172.21.15.60:0/2108423596 shutdown_connections 2023-08-24T10:07:12.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.723+0000 7f9602d9e640 1 --2- 172.21.15.60:0/2108423596 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f95e8037180 0x7f95e8039690 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:12.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.723+0000 7f9602d9e640 1 --2- 172.21.15.60:0/2108423596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95fc12f610 0x7f95fc0a0d10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:12.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.723+0000 7f9602d9e640 1 -- 172.21.15.60:0/2108423596 >> 172.21.15.60:0/2108423596 conn(0x7f95fc0a2500 msgr2=0x7f95fc099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:12.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.723+0000 7f9602d9e640 1 -- 172.21.15.60:0/2108423596 shutdown_connections 2023-08-24T10:07:12.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.723+0000 7f9602d9e640 1 -- 172.21.15.60:0/2108423596 wait complete. 2023-08-24T10:07:12.736 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:07:12.737 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:12.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.843+0000 7fcf9671e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:12.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.847+0000 7fcf9671e640 1 Processor -- start 2023-08-24T10:07:12.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.847+0000 7fcf9671e640 1 -- start start 2023-08-24T10:07:12.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.848+0000 7fcf9671e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcf9006d300 0x7fcf9012bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:12.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.848+0000 7fcf9671e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf90099440 con 0x7fcf9012c1c0 2023-08-24T10:07:12.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.848+0000 7fcf9671e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf9009a050 con 0x7fcf9006d300 2023-08-24T10:07:12.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.848+0000 7fcf8ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcf9006d300 0x7fcf9012bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:12.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.848+0000 7fcf8ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcf9006d300 0x7fcf9012bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41370/0 (socket says 172.21.15.60:41370) 2023-08-24T10:07:12.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.848+0000 7fcf8ffff640 1 -- 172.21.15.60:0/3426296221 learned_addr learned my addr 172.21.15.60:0/3426296221 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:12.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.848+0000 7fcf8f7fe640 1 --1- 172.21.15.60:0/3426296221 >> v1:172.21.15.60:6789/0 conn(0x7fcf9012c1c0 0x7fcf9012ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34952/0 (socket says 172.21.15.60:34952) 2023-08-24T10:07:12.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.848+0000 7fcf8ffff640 1 -- 172.21.15.60:0/3426296221 >> v1:172.21.15.60:6789/0 conn(0x7fcf9012c1c0 legacy=0x7fcf9012ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:12.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.848+0000 7fcf8ffff640 1 -- 172.21.15.60:0/3426296221 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9012b9d0 con 0x7fcf9006d300 2023-08-24T10:07:12.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.848+0000 7fcf8ffff640 1 --2- 172.21.15.60:0/3426296221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9006d300 0x7fcf9012bc80 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66a8235bb6f1482 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:12.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.848+0000 7fcf8effd640 1 -- 172.21.15.60:0/3426296221 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcf8000ac60 con 0x7fcf9006d300 2023-08-24T10:07:12.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.848+0000 7fcf8effd640 1 -- 172.21.15.60:0/3426296221 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fcf8000adc0 con 0x7fcf9006d300 2023-08-24T10:07:12.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.849+0000 7fcf8effd640 1 -- 172.21.15.60:0/3426296221 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcf8000f4d0 con 0x7fcf9006d300 2023-08-24T10:07:12.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.849+0000 7fcf9671e640 1 -- 172.21.15.60:0/3426296221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9006d300 msgr2=0x7fcf9012bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:12.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.849+0000 7fcf9671e640 1 --2- 172.21.15.60:0/3426296221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9006d300 0x7fcf9012bc80 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:12.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.849+0000 7fcf9671e640 1 -- 172.21.15.60:0/3426296221 shutdown_connections 2023-08-24T10:07:12.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.849+0000 7fcf9671e640 1 --2- 172.21.15.60:0/3426296221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9006d300 0x7fcf9012bc80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:12.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.849+0000 7fcf9671e640 1 -- 172.21.15.60:0/3426296221 >> 172.21.15.60:0/3426296221 conn(0x7fcf9009e110 msgr2=0x7fcf9009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:12.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.849+0000 7fcf9671e640 1 -- 172.21.15.60:0/3426296221 shutdown_connections 2023-08-24T10:07:12.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.849+0000 7fcf9671e640 1 -- 172.21.15.60:0/3426296221 wait complete. 2023-08-24T10:07:12.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.849+0000 7fcf9671e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:12.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.850+0000 7fcf9671e640 1 Processor -- start 2023-08-24T10:07:12.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.851+0000 7fcf9671e640 1 -- start start 2023-08-24T10:07:12.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.851+0000 7fcf9671e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9006d300 0x7fcf9006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:12.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.851+0000 7fcf9671e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf90089370 con 0x7fcf9006d300 2023-08-24T10:07:12.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.851+0000 7fcf8ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9006d300 0x7fcf9006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:12.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.851+0000 7fcf8ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9006d300 0x7fcf9006e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41378/0 (socket says 172.21.15.60:41378) 2023-08-24T10:07:12.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.851+0000 7fcf8ffff640 1 -- 172.21.15.60:0/1330631355 learned_addr learned my addr 172.21.15.60:0/1330631355 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:12.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.852+0000 7fcf8ffff640 1 -- 172.21.15.60:0/1330631355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf90096f10 con 0x7fcf9006d300 2023-08-24T10:07:12.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.852+0000 7fcf8ffff640 1 --2- 172.21.15.60:0/1330631355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9006d300 0x7fcf9006e9f0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.852+0000 7fcf8cff9640 1 -- 172.21.15.60:0/1330631355 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcf80015040 con 0x7fcf9006d300 2023-08-24T10:07:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.852+0000 7fcf8cff9640 1 -- 172.21.15.60:0/1330631355 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fcf80010540 con 0x7fcf9006d300 2023-08-24T10:07:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.852+0000 7fcf9671e640 1 -- 172.21.15.60:0/1330631355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf90089560 con 0x7fcf9006d300 2023-08-24T10:07:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.852+0000 7fcf8cff9640 1 -- 172.21.15.60:0/1330631355 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcf800107e0 con 0x7fcf9006d300 2023-08-24T10:07:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.852+0000 7fcf9671e640 1 -- 172.21.15.60:0/1330631355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf90088ff0 con 0x7fcf9006d300 2023-08-24T10:07:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.853+0000 7fcf8cff9640 1 -- 172.21.15.60:0/1330631355 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fcf80018040 con 0x7fcf9006d300 2023-08-24T10:07:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.853+0000 7fcf8cff9640 1 --2- 172.21.15.60:0/1330631355 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcf70037130 0x7fcf70039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:12.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.853+0000 7fcf8cff9640 1 -- 172.21.15.60:0/1330631355 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fcf80047a20 con 0x7fcf9006d300 2023-08-24T10:07:12.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.853+0000 7fcf9671e640 1 -- 172.21.15.60:0/1330631355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf9006e840 con 0x7fcf9006d300 2023-08-24T10:07:12.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.853+0000 7fcf8f7fe640 1 --2- 172.21.15.60:0/1330631355 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcf70037130 0x7fcf70039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:12.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.854+0000 7fcf8f7fe640 1 --2- 172.21.15.60:0/1330631355 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcf70037130 0x7fcf70039640 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:12.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:12.858+0000 7fcf8cff9640 1 -- 172.21.15.60:0/1330631355 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fcf80019360 con 0x7fcf9006d300 2023-08-24T10:07:13.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:13.029+0000 7fcf9671e640 1 -- 172.21.15.60:0/1330631355 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf9006da20 con 0x7fcf70037130 2023-08-24T10:07:13.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:13.031+0000 7fcf8cff9640 1 -- 172.21.15.60:0/1330631355 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7fcf78001420 con 0x7fcf70037130 2023-08-24T10:07:13.032 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:13.033 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:13.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:13.033+0000 7fcf9671e640 1 -- 172.21.15.60:0/1330631355 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcf70037130 msgr2=0x7fcf70039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:13.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:13.033+0000 7fcf9671e640 1 --2- 172.21.15.60:0/1330631355 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcf70037130 0x7fcf70039640 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:13.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:13.033+0000 7fcf9671e640 1 -- 172.21.15.60:0/1330631355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9006d300 msgr2=0x7fcf9006e9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:13.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:13.033+0000 7fcf9671e640 1 --2- 172.21.15.60:0/1330631355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9006d300 0x7fcf9006e9f0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:13.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:13.033+0000 7fcf9671e640 1 -- 172.21.15.60:0/1330631355 shutdown_connections 2023-08-24T10:07:13.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:13.033+0000 7fcf9671e640 1 --2- 172.21.15.60:0/1330631355 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcf70037130 0x7fcf70039640 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:13.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:13.033+0000 7fcf9671e640 1 --2- 172.21.15.60:0/1330631355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9006d300 0x7fcf9006e9f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:13.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:13.033+0000 7fcf9671e640 1 -- 172.21.15.60:0/1330631355 >> 172.21.15.60:0/1330631355 conn(0x7fcf9009e110 msgr2=0x7fcf9009bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:13.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:13.033+0000 7fcf9671e640 1 -- 172.21.15.60:0/1330631355 shutdown_connections 2023-08-24T10:07:13.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:13.033+0000 7fcf9671e640 1 -- 172.21.15.60:0/1330631355 wait complete. 2023-08-24T10:07:13.045 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2023-08-24T10:07:11.521325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:16.046 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:16.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.167+0000 7fd9d5eac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:16.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.171+0000 7fd9d5eac640 1 Processor -- start 2023-08-24T10:07:16.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.171+0000 7fd9d5eac640 1 -- start start 2023-08-24T10:07:16.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.171+0000 7fd9d5eac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd9d006d300 0x7fd9d012bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:16.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.171+0000 7fd9d5eac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d0099440 con 0x7fd9d012c1c0 2023-08-24T10:07:16.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.171+0000 7fd9d5eac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d009a050 con 0x7fd9d006d300 2023-08-24T10:07:16.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.172+0000 7fd9cf7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd9d006d300 0x7fd9d012bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:16.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.172+0000 7fd9cf7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd9d006d300 0x7fd9d012bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41382/0 (socket says 172.21.15.60:41382) 2023-08-24T10:07:16.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.172+0000 7fd9cf7fe640 1 -- 172.21.15.60:0/140614874 learned_addr learned my addr 172.21.15.60:0/140614874 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:16.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.172+0000 7fd9cf7fe640 1 -- 172.21.15.60:0/140614874 >> v1:172.21.15.60:6789/0 conn(0x7fd9d012c1c0 legacy=0x7fd9d012ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:16.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.172+0000 7fd9cf7fe640 1 -- 172.21.15.60:0/140614874 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d012b9d0 con 0x7fd9d006d300 2023-08-24T10:07:16.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.172+0000 7fd9cf7fe640 1 --2- 172.21.15.60:0/140614874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d006d300 0x7fd9d012bc80 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c706e3420f25cd server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:16.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.172+0000 7fd9ce7fc640 1 -- 172.21.15.60:0/140614874 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd9c0009dd0 con 0x7fd9d006d300 2023-08-24T10:07:16.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.172+0000 7fd9ce7fc640 1 -- 172.21.15.60:0/140614874 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd9c000e230 con 0x7fd9d006d300 2023-08-24T10:07:16.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.172+0000 7fd9ce7fc640 1 -- 172.21.15.60:0/140614874 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd9c000e4d0 con 0x7fd9d006d300 2023-08-24T10:07:16.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.173+0000 7fd9d5eac640 1 -- 172.21.15.60:0/140614874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d006d300 msgr2=0x7fd9d012bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:16.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.173+0000 7fd9d5eac640 1 --2- 172.21.15.60:0/140614874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d006d300 0x7fd9d012bc80 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:16.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.173+0000 7fd9d5eac640 1 -- 172.21.15.60:0/140614874 shutdown_connections 2023-08-24T10:07:16.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.173+0000 7fd9d5eac640 1 --2- 172.21.15.60:0/140614874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d006d300 0x7fd9d012bc80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:16.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.173+0000 7fd9d5eac640 1 -- 172.21.15.60:0/140614874 >> 172.21.15.60:0/140614874 conn(0x7fd9d009e110 msgr2=0x7fd9d009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:16.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.173+0000 7fd9d5eac640 1 -- 172.21.15.60:0/140614874 shutdown_connections 2023-08-24T10:07:16.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.173+0000 7fd9d5eac640 1 -- 172.21.15.60:0/140614874 wait complete. 2023-08-24T10:07:16.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.173+0000 7fd9d5eac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:16.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.174+0000 7fd9d5eac640 1 Processor -- start 2023-08-24T10:07:16.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.174+0000 7fd9d5eac640 1 -- start start 2023-08-24T10:07:16.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.175+0000 7fd9d5eac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d006d300 0x7fd9d006f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:16.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.175+0000 7fd9d5eac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d0071b20 con 0x7fd9d006d300 2023-08-24T10:07:16.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.175+0000 7fd9cf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d006d300 0x7fd9d006f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:16.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.175+0000 7fd9cf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d006d300 0x7fd9d006f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41392/0 (socket says 172.21.15.60:41392) 2023-08-24T10:07:16.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.175+0000 7fd9cf7fe640 1 -- 172.21.15.60:0/3571208091 learned_addr learned my addr 172.21.15.60:0/3571208091 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:16.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.176+0000 7fd9cf7fe640 1 -- 172.21.15.60:0/3571208091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d008dc00 con 0x7fd9d006d300 2023-08-24T10:07:16.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.176+0000 7fd9cf7fe640 1 --2- 172.21.15.60:0/3571208091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d006d300 0x7fd9d006f5b0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:16.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.176+0000 7fd9b7fff640 1 -- 172.21.15.60:0/3571208091 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd9c0014410 con 0x7fd9d006d300 2023-08-24T10:07:16.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.176+0000 7fd9b7fff640 1 -- 172.21.15.60:0/3571208091 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd9c0014570 con 0x7fd9d006d300 2023-08-24T10:07:16.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.176+0000 7fd9d5eac640 1 -- 172.21.15.60:0/3571208091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d0071810 con 0x7fd9d006d300 2023-08-24T10:07:16.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.176+0000 7fd9b7fff640 1 -- 172.21.15.60:0/3571208091 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd9c0014810 con 0x7fd9d006d300 2023-08-24T10:07:16.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.176+0000 7fd9d5eac640 1 -- 172.21.15.60:0/3571208091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d01295b0 con 0x7fd9d006d300 2023-08-24T10:07:16.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.177+0000 7fd9b7fff640 1 -- 172.21.15.60:0/3571208091 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fd9c00149c0 con 0x7fd9d006d300 2023-08-24T10:07:16.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.177+0000 7fd9d5eac640 1 -- 172.21.15.60:0/3571208091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d0129800 con 0x7fd9d006d300 2023-08-24T10:07:16.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.177+0000 7fd9b7fff640 1 --2- 172.21.15.60:0/3571208091 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd9bc037130 0x7fd9bc039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:16.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.177+0000 7fd9b7fff640 1 -- 172.21.15.60:0/3571208091 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fd9c0019040 con 0x7fd9d006d300 2023-08-24T10:07:16.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.177+0000 7fd9ceffd640 1 --2- 172.21.15.60:0/3571208091 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd9bc037130 0x7fd9bc039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:16.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.178+0000 7fd9ceffd640 1 --2- 172.21.15.60:0/3571208091 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd9bc037130 0x7fd9bc039640 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:16.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.182+0000 7fd9b7fff640 1 -- 172.21.15.60:0/3571208091 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fd9c000f8f0 con 0x7fd9d006d300 2023-08-24T10:07:16.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.354+0000 7fd9d5eac640 1 -- 172.21.15.60:0/3571208091 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd9d009b940 con 0x7fd9bc037130 2023-08-24T10:07:16.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.355+0000 7fd9b7fff640 1 -- 172.21.15.60:0/3571208091 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7fd9c4001650 con 0x7fd9bc037130 2023-08-24T10:07:16.357 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:16.357 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:16.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.357+0000 7fd9d5eac640 1 -- 172.21.15.60:0/3571208091 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd9bc037130 msgr2=0x7fd9bc039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:16.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.357+0000 7fd9d5eac640 1 --2- 172.21.15.60:0/3571208091 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd9bc037130 0x7fd9bc039640 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:16.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.357+0000 7fd9d5eac640 1 -- 172.21.15.60:0/3571208091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d006d300 msgr2=0x7fd9d006f5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:16.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.357+0000 7fd9d5eac640 1 --2- 172.21.15.60:0/3571208091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d006d300 0x7fd9d006f5b0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:16.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.357+0000 7fd9d5eac640 1 -- 172.21.15.60:0/3571208091 shutdown_connections 2023-08-24T10:07:16.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.357+0000 7fd9d5eac640 1 --2- 172.21.15.60:0/3571208091 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd9bc037130 0x7fd9bc039640 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:16.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.357+0000 7fd9d5eac640 1 --2- 172.21.15.60:0/3571208091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9d006d300 0x7fd9d006f5b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:16.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.357+0000 7fd9d5eac640 1 -- 172.21.15.60:0/3571208091 >> 172.21.15.60:0/3571208091 conn(0x7fd9d009e110 msgr2=0x7fd9d012e820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:16.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.357+0000 7fd9d5eac640 1 -- 172.21.15.60:0/3571208091 shutdown_connections 2023-08-24T10:07:16.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.357+0000 7fd9d5eac640 1 -- 172.21.15.60:0/3571208091 wait complete. 2023-08-24T10:07:16.369 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2023-08-24T10:07:15.522201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:16.372 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:16.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.479+0000 7fdd392ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:16.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.483+0000 7fdd392ec640 1 Processor -- start 2023-08-24T10:07:16.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.483+0000 7fdd392ec640 1 -- start start 2023-08-24T10:07:16.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.484+0000 7fdd392ec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdd34070110 0x7fdd341356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:16.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.484+0000 7fdd392ec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd34096f10 con 0x7fdd34071e40 2023-08-24T10:07:16.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.484+0000 7fdd392ec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd340977f0 con 0x7fdd34070110 2023-08-24T10:07:16.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.484+0000 7fdd327fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdd34070110 0x7fdd341356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:16.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.484+0000 7fdd327fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdd34070110 0x7fdd341356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41400/0 (socket says 172.21.15.60:41400) 2023-08-24T10:07:16.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.484+0000 7fdd327fc640 1 -- 172.21.15.60:0/3648083880 learned_addr learned my addr 172.21.15.60:0/3648083880 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:16.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.484+0000 7fdd327fc640 1 -- 172.21.15.60:0/3648083880 >> v1:172.21.15.60:6789/0 conn(0x7fdd34071e40 legacy=0x7fdd34096800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:16.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.484+0000 7fdd327fc640 1 -- 172.21.15.60:0/3648083880 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3409a280 con 0x7fdd34070110 2023-08-24T10:07:16.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.484+0000 7fdd327fc640 1 --2- 172.21.15.60:0/3648083880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd34070110 0x7fdd341356a0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec42b6746edf769 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:16.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.485+0000 7fdd31ffb640 1 -- 172.21.15.60:0/3648083880 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdd1c009dd0 con 0x7fdd34070110 2023-08-24T10:07:16.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.485+0000 7fdd31ffb640 1 -- 172.21.15.60:0/3648083880 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fdd1c00e230 con 0x7fdd34070110 2023-08-24T10:07:16.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.485+0000 7fdd31ffb640 1 -- 172.21.15.60:0/3648083880 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdd1c00e4d0 con 0x7fdd34070110 2023-08-24T10:07:16.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.485+0000 7fdd392ec640 1 -- 172.21.15.60:0/3648083880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd34070110 msgr2=0x7fdd341356a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:16.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.485+0000 7fdd392ec640 1 --2- 172.21.15.60:0/3648083880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd34070110 0x7fdd341356a0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:16.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.485+0000 7fdd392ec640 1 -- 172.21.15.60:0/3648083880 shutdown_connections 2023-08-24T10:07:16.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.485+0000 7fdd392ec640 1 --2- 172.21.15.60:0/3648083880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd34070110 0x7fdd341356a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:16.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.485+0000 7fdd392ec640 1 -- 172.21.15.60:0/3648083880 >> 172.21.15.60:0/3648083880 conn(0x7fdd3409e110 msgr2=0x7fdd3409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:16.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.485+0000 7fdd392ec640 1 -- 172.21.15.60:0/3648083880 shutdown_connections 2023-08-24T10:07:16.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.485+0000 7fdd392ec640 1 -- 172.21.15.60:0/3648083880 wait complete. 2023-08-24T10:07:16.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.486+0000 7fdd392ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:16.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.487+0000 7fdd392ec640 1 Processor -- start 2023-08-24T10:07:16.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.487+0000 7fdd392ec640 1 -- start start 2023-08-24T10:07:16.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.487+0000 7fdd392ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd34070110 0x7fdd34093460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:16.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.487+0000 7fdd392ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3408ff10 con 0x7fdd34070110 2023-08-24T10:07:16.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.487+0000 7fdd32ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd34070110 0x7fdd34093460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:16.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.487+0000 7fdd32ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd34070110 0x7fdd34093460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41412/0 (socket says 172.21.15.60:41412) 2023-08-24T10:07:16.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.487+0000 7fdd32ffd640 1 -- 172.21.15.60:0/542604004 learned_addr learned my addr 172.21.15.60:0/542604004 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:16.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.488+0000 7fdd32ffd640 1 -- 172.21.15.60:0/542604004 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd34090680 con 0x7fdd34070110 2023-08-24T10:07:16.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.488+0000 7fdd32ffd640 1 --2- 172.21.15.60:0/542604004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd34070110 0x7fdd34093460 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:16.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.488+0000 7fdd2b7fe640 1 -- 172.21.15.60:0/542604004 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdd2400cbd0 con 0x7fdd34070110 2023-08-24T10:07:16.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.488+0000 7fdd2b7fe640 1 -- 172.21.15.60:0/542604004 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fdd2400cd30 con 0x7fdd34070110 2023-08-24T10:07:16.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.488+0000 7fdd392ec640 1 -- 172.21.15.60:0/542604004 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd34092e80 con 0x7fdd34070110 2023-08-24T10:07:16.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.488+0000 7fdd2b7fe640 1 -- 172.21.15.60:0/542604004 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdd2400f230 con 0x7fdd34070110 2023-08-24T10:07:16.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.488+0000 7fdd392ec640 1 -- 172.21.15.60:0/542604004 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd34092b70 con 0x7fdd34070110 2023-08-24T10:07:16.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.489+0000 7fdd392ec640 1 -- 172.21.15.60:0/542604004 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd34061800 con 0x7fdd34070110 2023-08-24T10:07:16.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.489+0000 7fdd2b7fe640 1 -- 172.21.15.60:0/542604004 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fdd2400f3e0 con 0x7fdd34070110 2023-08-24T10:07:16.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.489+0000 7fdd2b7fe640 1 --2- 172.21.15.60:0/542604004 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fdd14037130 0x7fdd14039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:16.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.489+0000 7fdd2b7fe640 1 -- 172.21.15.60:0/542604004 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fdd24046dc0 con 0x7fdd34070110 2023-08-24T10:07:16.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.490+0000 7fdd327fc640 1 --2- 172.21.15.60:0/542604004 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fdd14037130 0x7fdd14039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:16.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.490+0000 7fdd327fc640 1 --2- 172.21.15.60:0/542604004 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fdd14037130 0x7fdd14039640 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:16.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.494+0000 7fdd2b7fe640 1 -- 172.21.15.60:0/542604004 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fdd2400ab00 con 0x7fdd34070110 2023-08-24T10:07:16.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.664+0000 7fdd392ec640 1 -- 172.21.15.60:0/542604004 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdd340932f0 con 0x7fdd14037130 2023-08-24T10:07:16.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.666+0000 7fdd2b7fe640 1 -- 172.21.15.60:0/542604004 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7fdd340932f0 con 0x7fdd14037130 2023-08-24T10:07:16.667 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:16.668 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:16.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.667+0000 7fdd392ec640 1 -- 172.21.15.60:0/542604004 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fdd14037130 msgr2=0x7fdd14039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.667+0000 7fdd392ec640 1 --2- 172.21.15.60:0/542604004 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fdd14037130 0x7fdd14039640 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.667+0000 7fdd392ec640 1 -- 172.21.15.60:0/542604004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd34070110 msgr2=0x7fdd34093460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.667+0000 7fdd392ec640 1 --2- 172.21.15.60:0/542604004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd34070110 0x7fdd34093460 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.667+0000 7fdd392ec640 1 -- 172.21.15.60:0/542604004 shutdown_connections 2023-08-24T10:07:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.667+0000 7fdd392ec640 1 --2- 172.21.15.60:0/542604004 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fdd14037130 0x7fdd14039640 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.667+0000 7fdd392ec640 1 --2- 172.21.15.60:0/542604004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd34070110 0x7fdd34093460 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:16.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.667+0000 7fdd392ec640 1 -- 172.21.15.60:0/542604004 >> 172.21.15.60:0/542604004 conn(0x7fdd3409e110 msgr2=0x7fdd34098490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:16.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.667+0000 7fdd392ec640 1 -- 172.21.15.60:0/542604004 shutdown_connections 2023-08-24T10:07:16.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.667+0000 7fdd392ec640 1 -- 172.21.15.60:0/542604004 wait complete. 2023-08-24T10:07:16.679 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2023-08-24T10:07:15.522201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:16.680 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:07:16.680 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:16.681 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:07:16.681 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:16.681 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:07:16.681 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:16.682 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:07:16.682 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:16.682 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:07:16.683 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:16.683 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:07:16.684 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:16.684 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:07:16.684 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:16.685 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:07:16.685 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:16.687 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:07:16.687 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:16.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:07:16.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.794+0000 7effb586c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:16.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.797+0000 7effb586c640 1 Processor -- start 2023-08-24T10:07:16.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.797+0000 7effb586c640 1 -- start start 2023-08-24T10:07:16.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.798+0000 7effb586c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7effb0132680 0x7effb0134aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:16.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.798+0000 7effb586c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb006dfc0 con 0x7effb0070c30 2023-08-24T10:07:16.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.798+0000 7effb586c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb006d9b0 con 0x7effb0132680 2023-08-24T10:07:16.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.798+0000 7effae7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7effb0132680 0x7effb0134aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:16.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.798+0000 7effae7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7effb0132680 0x7effb0134aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39186/0 (socket says 172.21.15.60:39186) 2023-08-24T10:07:16.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.798+0000 7effae7fc640 1 -- 172.21.15.60:0/3581035770 learned_addr learned my addr 172.21.15.60:0/3581035770 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:16.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.798+0000 7effaeffd640 1 --1- 172.21.15.60:0/3581035770 >> v1:172.21.15.60:6789/0 conn(0x7effb0070c30 0x7effb0071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56436/0 (socket says 172.21.15.60:56436) 2023-08-24T10:07:16.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.798+0000 7effae7fc640 1 -- 172.21.15.60:0/3581035770 >> v1:172.21.15.60:6789/0 conn(0x7effb0070c30 legacy=0x7effb0071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.798+0000 7effae7fc640 1 -- 172.21.15.60:0/3581035770 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb006e6e0 con 0x7effb0132680 2023-08-24T10:07:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.798+0000 7effae7fc640 1 --2- 172.21.15.60:0/3581035770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effb0132680 0x7effb0134aa0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da0c21a53cd11cc server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.798+0000 7effadffb640 1 -- 172.21.15.60:0/3581035770 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7effa4009dd0 con 0x7effb0132680 2023-08-24T10:07:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.798+0000 7effadffb640 1 -- 172.21.15.60:0/3581035770 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7effa400e230 con 0x7effb0132680 2023-08-24T10:07:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.798+0000 7effadffb640 1 -- 172.21.15.60:0/3581035770 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7effa400e4d0 con 0x7effb0132680 2023-08-24T10:07:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.799+0000 7effb586c640 1 -- 172.21.15.60:0/3581035770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effb0132680 msgr2=0x7effb0134aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:16.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.799+0000 7effb586c640 1 --2- 172.21.15.60:0/3581035770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effb0132680 0x7effb0134aa0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.799+0000 7effb586c640 1 -- 172.21.15.60:0/3581035770 shutdown_connections 2023-08-24T10:07:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.799+0000 7effb586c640 1 --2- 172.21.15.60:0/3581035770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effb0132680 0x7effb0134aa0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.799+0000 7effb586c640 1 -- 172.21.15.60:0/3581035770 >> 172.21.15.60:0/3581035770 conn(0x7effb009e110 msgr2=0x7effb009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.799+0000 7effb586c640 1 -- 172.21.15.60:0/3581035770 shutdown_connections 2023-08-24T10:07:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.799+0000 7effb586c640 1 -- 172.21.15.60:0/3581035770 wait complete. 2023-08-24T10:07:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.799+0000 7effb586c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.800+0000 7effb586c640 1 Processor -- start 2023-08-24T10:07:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.801+0000 7effb586c640 1 -- start start 2023-08-24T10:07:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.801+0000 7effb586c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effb0070c30 0x7effb012c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.801+0000 7effb586c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb012f4a0 con 0x7effb0070c30 2023-08-24T10:07:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.801+0000 7effaeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effb0070c30 0x7effb012c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.801+0000 7effaeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effb0070c30 0x7effb012c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39196/0 (socket says 172.21.15.60:39196) 2023-08-24T10:07:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.801+0000 7effaeffd640 1 -- 172.21.15.60:0/3684061972 learned_addr learned my addr 172.21.15.60:0/3684061972 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.802+0000 7effaeffd640 1 -- 172.21.15.60:0/3684061972 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb01300b0 con 0x7effb0070c30 2023-08-24T10:07:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.802+0000 7effaeffd640 1 --2- 172.21.15.60:0/3684061972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effb0070c30 0x7effb012c1a0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.802+0000 7eff97fff640 1 -- 172.21.15.60:0/3684061972 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7effa0005c90 con 0x7effb0070c30 2023-08-24T10:07:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.802+0000 7eff97fff640 1 -- 172.21.15.60:0/3684061972 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7effa0005df0 con 0x7effb0070c30 2023-08-24T10:07:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.802+0000 7effb586c640 1 -- 172.21.15.60:0/3684061972 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb0098b50 con 0x7effb0070c30 2023-08-24T10:07:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.802+0000 7eff97fff640 1 -- 172.21.15.60:0/3684061972 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7effa000cba0 con 0x7effb0070c30 2023-08-24T10:07:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.802+0000 7effb586c640 1 -- 172.21.15.60:0/3684061972 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb012dd40 con 0x7effb0070c30 2023-08-24T10:07:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.803+0000 7eff97fff640 1 -- 172.21.15.60:0/3684061972 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7effa00070b0 con 0x7effb0070c30 2023-08-24T10:07:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.803+0000 7effb586c640 1 -- 172.21.15.60:0/3684061972 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb0061800 con 0x7effb0070c30 2023-08-24T10:07:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.803+0000 7eff97fff640 1 --2- 172.21.15.60:0/3684061972 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7eff9c037130 0x7eff9c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.803+0000 7eff97fff640 1 -- 172.21.15.60:0/3684061972 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7effa0046750 con 0x7effb0070c30 2023-08-24T10:07:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.804+0000 7effae7fc640 1 --2- 172.21.15.60:0/3684061972 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7eff9c037130 0x7eff9c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.804+0000 7effae7fc640 1 --2- 172.21.15.60:0/3684061972 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7eff9c037130 0x7eff9c039640 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:16.808+0000 7eff97fff640 1 -- 172.21.15.60:0/3684061972 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7effa000d980 con 0x7effb0070c30 2023-08-24T10:07:17.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.053+0000 7effb586c640 1 -- 172.21.15.60:0/3684061972 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7effb00706e0 con 0x7effb0070c30 2023-08-24T10:07:17.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.053+0000 7eff97fff640 1 -- 172.21.15.60:0/3684061972 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7effa000d150 con 0x7effb0070c30 2023-08-24T10:07:17.057 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:17.057 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":147,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:07:17.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.055+0000 7effb586c640 1 -- 172.21.15.60:0/3684061972 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7eff9c037130 msgr2=0x7eff9c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:17.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.055+0000 7effb586c640 1 --2- 172.21.15.60:0/3684061972 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7eff9c037130 0x7eff9c039640 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:17.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.055+0000 7effb586c640 1 -- 172.21.15.60:0/3684061972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effb0070c30 msgr2=0x7effb012c1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:17.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.055+0000 7effb586c640 1 --2- 172.21.15.60:0/3684061972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effb0070c30 0x7effb012c1a0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:17.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.055+0000 7effb586c640 1 -- 172.21.15.60:0/3684061972 shutdown_connections 2023-08-24T10:07:17.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.055+0000 7effb586c640 1 --2- 172.21.15.60:0/3684061972 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7eff9c037130 0x7eff9c039640 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:17.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.055+0000 7effb586c640 1 --2- 172.21.15.60:0/3684061972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effb0070c30 0x7effb012c1a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:17.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.055+0000 7effb586c640 1 -- 172.21.15.60:0/3684061972 >> 172.21.15.60:0/3684061972 conn(0x7effb009e110 msgr2=0x7effb01b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:17.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.055+0000 7effb586c640 1 -- 172.21.15.60:0/3684061972 shutdown_connections 2023-08-24T10:07:17.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.055+0000 7effb586c640 1 -- 172.21.15.60:0/3684061972 wait complete. 2023-08-24T10:07:17.067 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:07:17.067 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:17.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.174+0000 7f93fd86c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:17.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.178+0000 7f93fd86c640 1 Processor -- start 2023-08-24T10:07:17.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.178+0000 7f93fd86c640 1 -- start start 2023-08-24T10:07:17.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.179+0000 7f93fd86c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93f812d040 0x7f93f81358b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:17.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.179+0000 7f93fd86c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f80709e0 con 0x7f93f806fa30 2023-08-24T10:07:17.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.179+0000 7f93fd86c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93f812d8d0 con 0x7f93f812d040 2023-08-24T10:07:17.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.179+0000 7f93f67fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93f812d040 0x7f93f81358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:17.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.179+0000 7f93f67fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93f812d040 0x7f93f81358b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39200/0 (socket says 172.21.15.60:39200) 2023-08-24T10:07:17.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.179+0000 7f93f67fc640 1 -- 172.21.15.60:0/1621340532 learned_addr learned my addr 172.21.15.60:0/1621340532 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:17.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.179+0000 7f93f67fc640 1 -- 172.21.15.60:0/1621340532 >> v1:172.21.15.60:6789/0 conn(0x7f93f806fa30 legacy=0x7f93f8098cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:17.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.179+0000 7f93f67fc640 1 -- 172.21.15.60:0/1621340532 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f812d5c0 con 0x7f93f812d040 2023-08-24T10:07:17.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.180+0000 7f93f67fc640 1 --2- 172.21.15.60:0/1621340532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93f812d040 0x7f93f81358b0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c44a8a5ae27cbc3 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:17.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.180+0000 7f93f5ffb640 1 -- 172.21.15.60:0/1621340532 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f93e0009dd0 con 0x7f93f812d040 2023-08-24T10:07:17.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.180+0000 7f93f5ffb640 1 -- 172.21.15.60:0/1621340532 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f93e000e230 con 0x7f93f812d040 2023-08-24T10:07:17.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.180+0000 7f93f5ffb640 1 -- 172.21.15.60:0/1621340532 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f93e000e4d0 con 0x7f93f812d040 2023-08-24T10:07:17.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.180+0000 7f93fd86c640 1 -- 172.21.15.60:0/1621340532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93f812d040 msgr2=0x7f93f81358b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:17.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.180+0000 7f93fd86c640 1 --2- 172.21.15.60:0/1621340532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93f812d040 0x7f93f81358b0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:17.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.180+0000 7f93fd86c640 1 -- 172.21.15.60:0/1621340532 shutdown_connections 2023-08-24T10:07:17.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.180+0000 7f93fd86c640 1 --2- 172.21.15.60:0/1621340532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93f812d040 0x7f93f81358b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:17.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.180+0000 7f93fd86c640 1 -- 172.21.15.60:0/1621340532 >> 172.21.15.60:0/1621340532 conn(0x7f93f80a2700 msgr2=0x7f93f80a2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:17.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.180+0000 7f93fd86c640 1 -- 172.21.15.60:0/1621340532 shutdown_connections 2023-08-24T10:07:17.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.180+0000 7f93fd86c640 1 -- 172.21.15.60:0/1621340532 wait complete. 2023-08-24T10:07:17.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.181+0000 7f93fd86c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:17.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.181+0000 7f93fd86c640 1 Processor -- start 2023-08-24T10:07:17.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.182+0000 7f93fd86c640 1 -- start start 2023-08-24T10:07:17.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.182+0000 7f93fd86c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93f806fa30 0x7f93f8119190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:17.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.182+0000 7f93fd86c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f81b9c00 con 0x7f93f806fa30 2023-08-24T10:07:17.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.182+0000 7f93f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93f806fa30 0x7f93f8119190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:17.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.182+0000 7f93f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93f806fa30 0x7f93f8119190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39208/0 (socket says 172.21.15.60:39208) 2023-08-24T10:07:17.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.182+0000 7f93f6ffd640 1 -- 172.21.15.60:0/1054955792 learned_addr learned my addr 172.21.15.60:0/1054955792 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:17.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.183+0000 7f93f6ffd640 1 -- 172.21.15.60:0/1054955792 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f8119cf0 con 0x7f93f806fa30 2023-08-24T10:07:17.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.183+0000 7f93f6ffd640 1 --2- 172.21.15.60:0/1054955792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93f806fa30 0x7f93f8119190 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:17.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.183+0000 7f93effff640 1 -- 172.21.15.60:0/1054955792 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f93f000c7e0 con 0x7f93f806fa30 2023-08-24T10:07:17.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.183+0000 7f93effff640 1 -- 172.21.15.60:0/1054955792 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f93f000c940 con 0x7f93f806fa30 2023-08-24T10:07:17.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.183+0000 7f93fd86c640 1 -- 172.21.15.60:0/1054955792 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f8130980 con 0x7f93f806fa30 2023-08-24T10:07:17.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.183+0000 7f93effff640 1 -- 172.21.15.60:0/1054955792 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f93f000cbe0 con 0x7f93f806fa30 2023-08-24T10:07:17.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.183+0000 7f93fd86c640 1 -- 172.21.15.60:0/1054955792 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f8119820 con 0x7f93f806fa30 2023-08-24T10:07:17.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.184+0000 7f93effff640 1 -- 172.21.15.60:0/1054955792 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f93f0011310 con 0x7f93f806fa30 2023-08-24T10:07:17.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.184+0000 7f93effff640 1 --2- 172.21.15.60:0/1054955792 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f93d80370e0 0x7f93d80395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.184+0000 7f93fd86c640 1 -- 172.21.15.60:0/1054955792 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f8061800 con 0x7f93f806fa30 2023-08-24T10:07:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.184+0000 7f93effff640 1 -- 172.21.15.60:0/1054955792 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f93f0046a90 con 0x7f93f806fa30 2023-08-24T10:07:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.185+0000 7f93f67fc640 1 --2- 172.21.15.60:0/1054955792 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f93d80370e0 0x7f93d80395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.185+0000 7f93f67fc640 1 --2- 172.21.15.60:0/1054955792 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f93d80370e0 0x7f93d80395f0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.189+0000 7f93effff640 1 -- 172.21.15.60:0/1054955792 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f93f000ad00 con 0x7f93f806fa30 2023-08-24T10:07:17.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.363+0000 7f93fd86c640 1 -- 172.21.15.60:0/1054955792 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f93f81310a0 con 0x7f93d80370e0 2023-08-24T10:07:17.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.365+0000 7f93effff640 1 -- 172.21.15.60:0/1054955792 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f93f81310a0 con 0x7f93d80370e0 2023-08-24T10:07:17.366 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:17.367 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:17.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.367+0000 7f93fd86c640 1 -- 172.21.15.60:0/1054955792 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f93d80370e0 msgr2=0x7f93d80395f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:17.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.367+0000 7f93fd86c640 1 --2- 172.21.15.60:0/1054955792 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f93d80370e0 0x7f93d80395f0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:17.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.367+0000 7f93fd86c640 1 -- 172.21.15.60:0/1054955792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93f806fa30 msgr2=0x7f93f8119190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:17.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.367+0000 7f93fd86c640 1 --2- 172.21.15.60:0/1054955792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93f806fa30 0x7f93f8119190 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:17.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.367+0000 7f93fd86c640 1 -- 172.21.15.60:0/1054955792 shutdown_connections 2023-08-24T10:07:17.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.367+0000 7f93fd86c640 1 --2- 172.21.15.60:0/1054955792 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f93d80370e0 0x7f93d80395f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:17.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.367+0000 7f93fd86c640 1 --2- 172.21.15.60:0/1054955792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93f806fa30 0x7f93f8119190 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:17.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.367+0000 7f93fd86c640 1 -- 172.21.15.60:0/1054955792 >> 172.21.15.60:0/1054955792 conn(0x7f93f80a2700 msgr2=0x7f93f81b87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:17.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.367+0000 7f93fd86c640 1 -- 172.21.15.60:0/1054955792 shutdown_connections 2023-08-24T10:07:17.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:17.367+0000 7f93fd86c640 1 -- 172.21.15.60:0/1054955792 wait complete. 2023-08-24T10:07:17.378 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2023-08-24T10:07:15.522201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:17.542 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~130s 2023-08-24T10:07:17.543 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~130s 2023-08-24T10:07:17.543 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~130s 2023-08-24T10:07:17.543 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~130s 2023-08-24T10:07:20.380 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:20.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.501+0000 7f069e66c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:20.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.505+0000 7f069e66c640 1 Processor -- start 2023-08-24T10:07:20.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.505+0000 7f069e66c640 1 -- start start 2023-08-24T10:07:20.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.505+0000 7f069e66c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0698071e40 0x7f0698096800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:20.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.505+0000 7f069e66c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06980977f0 con 0x7f069806f100 2023-08-24T10:07:20.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.505+0000 7f069e66c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069809a280 con 0x7f0698071e40 2023-08-24T10:07:20.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.506+0000 7f0697fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0698071e40 0x7f0698096800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:20.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.506+0000 7f0697fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0698071e40 0x7f0698096800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39216/0 (socket says 172.21.15.60:39216) 2023-08-24T10:07:20.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.506+0000 7f0697fff640 1 -- 172.21.15.60:0/2232546835 learned_addr learned my addr 172.21.15.60:0/2232546835 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:20.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.506+0000 7f0697fff640 1 -- 172.21.15.60:0/2232546835 >> v1:172.21.15.60:6789/0 conn(0x7f069806f100 legacy=0x7f069806f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:20.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.506+0000 7f0697fff640 1 -- 172.21.15.60:0/2232546835 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069809a6b0 con 0x7f0698071e40 2023-08-24T10:07:20.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.506+0000 7f0697fff640 1 --2- 172.21.15.60:0/2232546835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0698071e40 0x7f0698096800 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01c8d43848d21d9 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.506+0000 7f0696ffd640 1 -- 172.21.15.60:0/2232546835 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0688009dd0 con 0x7f0698071e40 2023-08-24T10:07:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.506+0000 7f0696ffd640 1 -- 172.21.15.60:0/2232546835 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f068800e230 con 0x7f0698071e40 2023-08-24T10:07:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.506+0000 7f0696ffd640 1 -- 172.21.15.60:0/2232546835 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f068800e4d0 con 0x7f0698071e40 2023-08-24T10:07:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.507+0000 7f069e66c640 1 -- 172.21.15.60:0/2232546835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0698071e40 msgr2=0x7f0698096800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.507+0000 7f069e66c640 1 --2- 172.21.15.60:0/2232546835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0698071e40 0x7f0698096800 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:20.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.507+0000 7f069e66c640 1 -- 172.21.15.60:0/2232546835 shutdown_connections 2023-08-24T10:07:20.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.507+0000 7f069e66c640 1 --2- 172.21.15.60:0/2232546835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0698071e40 0x7f0698096800 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:20.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.507+0000 7f069e66c640 1 -- 172.21.15.60:0/2232546835 >> 172.21.15.60:0/2232546835 conn(0x7f069809e110 msgr2=0x7f069809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:20.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.507+0000 7f069e66c640 1 -- 172.21.15.60:0/2232546835 shutdown_connections 2023-08-24T10:07:20.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.507+0000 7f069e66c640 1 -- 172.21.15.60:0/2232546835 wait complete. 2023-08-24T10:07:20.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.507+0000 7f069e66c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:20.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.508+0000 7f069e66c640 1 Processor -- start 2023-08-24T10:07:20.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.508+0000 7f069e66c640 1 -- start start 2023-08-24T10:07:20.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.509+0000 7f069e66c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f069806f100 0x7f06981c19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:20.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.509+0000 7f069e66c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069808ffe0 con 0x7f069806f100 2023-08-24T10:07:20.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.509+0000 7f0697fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f069806f100 0x7f06981c19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:20.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.509+0000 7f0697fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f069806f100 0x7f06981c19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39226/0 (socket says 172.21.15.60:39226) 2023-08-24T10:07:20.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.509+0000 7f0697fff640 1 -- 172.21.15.60:0/2714458295 learned_addr learned my addr 172.21.15.60:0/2714458295 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:20.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.509+0000 7f0697fff640 1 -- 172.21.15.60:0/2714458295 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06981c14b0 con 0x7f069806f100 2023-08-24T10:07:20.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.510+0000 7f0697fff640 1 --2- 172.21.15.60:0/2714458295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f069806f100 0x7f06981c19f0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:20.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.510+0000 7f0694ff9640 1 -- 172.21.15.60:0/2714458295 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0688014540 con 0x7f069806f100 2023-08-24T10:07:20.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.510+0000 7f0694ff9640 1 -- 172.21.15.60:0/2714458295 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f06880146a0 con 0x7f069806f100 2023-08-24T10:07:20.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.510+0000 7f069e66c640 1 -- 172.21.15.60:0/2714458295 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0698090720 con 0x7f069806f100 2023-08-24T10:07:20.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.510+0000 7f0694ff9640 1 -- 172.21.15.60:0/2714458295 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0688014940 con 0x7f069806f100 2023-08-24T10:07:20.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.510+0000 7f069e66c640 1 -- 172.21.15.60:0/2714458295 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0698092c10 con 0x7f069806f100 2023-08-24T10:07:20.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.511+0000 7f0694ff9640 1 -- 172.21.15.60:0/2714458295 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f0688014b40 con 0x7f069806f100 2023-08-24T10:07:20.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.511+0000 7f069e66c640 1 -- 172.21.15.60:0/2714458295 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0698061800 con 0x7f069806f100 2023-08-24T10:07:20.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.511+0000 7f0694ff9640 1 --2- 172.21.15.60:0/2714458295 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f06780370e0 0x7f06780395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:20.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.511+0000 7f0694ff9640 1 -- 172.21.15.60:0/2714458295 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f0688019040 con 0x7f069806f100 2023-08-24T10:07:20.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.511+0000 7f06977fe640 1 --2- 172.21.15.60:0/2714458295 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f06780370e0 0x7f06780395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:20.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.512+0000 7f06977fe640 1 --2- 172.21.15.60:0/2714458295 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f06780370e0 0x7f06780395f0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:20.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.516+0000 7f0694ff9640 1 -- 172.21.15.60:0/2714458295 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f0688018190 con 0x7f069806f100 2023-08-24T10:07:20.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.689+0000 7f069e66c640 1 -- 172.21.15.60:0/2714458295 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0698097a80 con 0x7f06780370e0 2023-08-24T10:07:20.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.690+0000 7f0694ff9640 1 -- 172.21.15.60:0/2714458295 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f0680001650 con 0x7f06780370e0 2023-08-24T10:07:20.692 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:20.692 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:20.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.692+0000 7f069e66c640 1 -- 172.21.15.60:0/2714458295 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f06780370e0 msgr2=0x7f06780395f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:20.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.692+0000 7f069e66c640 1 --2- 172.21.15.60:0/2714458295 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f06780370e0 0x7f06780395f0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:20.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.692+0000 7f069e66c640 1 -- 172.21.15.60:0/2714458295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f069806f100 msgr2=0x7f06981c19f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:20.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.692+0000 7f069e66c640 1 --2- 172.21.15.60:0/2714458295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f069806f100 0x7f06981c19f0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:20.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.692+0000 7f069e66c640 1 -- 172.21.15.60:0/2714458295 shutdown_connections 2023-08-24T10:07:20.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.692+0000 7f069e66c640 1 --2- 172.21.15.60:0/2714458295 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f06780370e0 0x7f06780395f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:20.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.692+0000 7f069e66c640 1 --2- 172.21.15.60:0/2714458295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f069806f100 0x7f06981c19f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:20.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.692+0000 7f069e66c640 1 -- 172.21.15.60:0/2714458295 >> 172.21.15.60:0/2714458295 conn(0x7f069809e110 msgr2=0x7f06980985b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:20.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.692+0000 7f069e66c640 1 -- 172.21.15.60:0/2714458295 shutdown_connections 2023-08-24T10:07:20.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.692+0000 7f069e66c640 1 -- 172.21.15.60:0/2714458295 wait complete. 2023-08-24T10:07:20.703 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2023-08-24T10:07:19.523065+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:20.705 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:20.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.812+0000 7f08e499e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:20.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.815+0000 7f08e499e640 1 Processor -- start 2023-08-24T10:07:20.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.815+0000 7f08e499e640 1 -- start start 2023-08-24T10:07:20.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.816+0000 7f08e499e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08e0132630 0x7f08e0134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:20.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.816+0000 7f08e499e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e006e320 con 0x7f08e0096860 2023-08-24T10:07:20.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.816+0000 7f08e499e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e006db70 con 0x7f08e0132630 2023-08-24T10:07:20.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.816+0000 7f08d5d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08e0132630 0x7f08e0134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:20.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.816+0000 7f08d5d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08e0132630 0x7f08e0134a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39242/0 (socket says 172.21.15.60:39242) 2023-08-24T10:07:20.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.816+0000 7f08d5d9b640 1 -- 172.21.15.60:0/1063507225 learned_addr learned my addr 172.21.15.60:0/1063507225 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.816+0000 7f08de59c640 1 --1- 172.21.15.60:0/1063507225 >> v1:172.21.15.60:6789/0 conn(0x7f08e0096860 0x7f08e0096c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56458/0 (socket says 172.21.15.60:56458) 2023-08-24T10:07:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.817+0000 7f08d5d9b640 1 -- 172.21.15.60:0/1063507225 >> v1:172.21.15.60:6789/0 conn(0x7f08e0096860 legacy=0x7f08e0096c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.817+0000 7f08d5d9b640 1 -- 172.21.15.60:0/1063507225 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e006cef0 con 0x7f08e0132630 2023-08-24T10:07:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.817+0000 7f08d5d9b640 1 --2- 172.21.15.60:0/1063507225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08e0132630 0x7f08e0134a50 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7137e77455c2bf7 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.817+0000 7f08ddd9b640 1 -- 172.21.15.60:0/1063507225 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f08c8009dd0 con 0x7f08e0132630 2023-08-24T10:07:20.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.817+0000 7f08ddd9b640 1 -- 172.21.15.60:0/1063507225 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f08c800e230 con 0x7f08e0132630 2023-08-24T10:07:20.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.817+0000 7f08ddd9b640 1 -- 172.21.15.60:0/1063507225 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f08c800e4d0 con 0x7f08e0132630 2023-08-24T10:07:20.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.818+0000 7f08e499e640 1 -- 172.21.15.60:0/1063507225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08e0132630 msgr2=0x7f08e0134a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:20.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.818+0000 7f08e499e640 1 --2- 172.21.15.60:0/1063507225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08e0132630 0x7f08e0134a50 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:20.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.818+0000 7f08e499e640 1 -- 172.21.15.60:0/1063507225 shutdown_connections 2023-08-24T10:07:20.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.818+0000 7f08e499e640 1 --2- 172.21.15.60:0/1063507225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08e0132630 0x7f08e0134a50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:20.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.818+0000 7f08e499e640 1 -- 172.21.15.60:0/1063507225 >> 172.21.15.60:0/1063507225 conn(0x7f08e009e110 msgr2=0x7f08e009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:20.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.818+0000 7f08e499e640 1 -- 172.21.15.60:0/1063507225 shutdown_connections 2023-08-24T10:07:20.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.818+0000 7f08e499e640 1 -- 172.21.15.60:0/1063507225 wait complete. 2023-08-24T10:07:20.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.818+0000 7f08e499e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:20.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.819+0000 7f08e499e640 1 Processor -- start 2023-08-24T10:07:20.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.819+0000 7f08e499e640 1 -- start start 2023-08-24T10:07:20.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.820+0000 7f08e499e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08e0096860 0x7f08e01c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:20.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.820+0000 7f08e499e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e0098ce0 con 0x7f08e0096860 2023-08-24T10:07:20.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.820+0000 7f08de59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08e0096860 0x7f08e01c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:20.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.820+0000 7f08de59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08e0096860 0x7f08e01c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39244/0 (socket says 172.21.15.60:39244) 2023-08-24T10:07:20.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.820+0000 7f08de59c640 1 -- 172.21.15.60:0/1914825394 learned_addr learned my addr 172.21.15.60:0/1914825394 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:20.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.820+0000 7f08de59c640 1 -- 172.21.15.60:0/1914825394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e012b2a0 con 0x7f08e0096860 2023-08-24T10:07:20.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.820+0000 7f08de59c640 1 --2- 172.21.15.60:0/1914825394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08e0096860 0x7f08e01c52a0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:20.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.820+0000 7f08d77fe640 1 -- 172.21.15.60:0/1914825394 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f08d00071e0 con 0x7f08e0096860 2023-08-24T10:07:20.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.820+0000 7f08d77fe640 1 -- 172.21.15.60:0/1914825394 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f08d0007340 con 0x7f08e0096860 2023-08-24T10:07:20.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.820+0000 7f08e499e640 1 -- 172.21.15.60:0/1914825394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e012b9e0 con 0x7f08e0096860 2023-08-24T10:07:20.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.820+0000 7f08d77fe640 1 -- 172.21.15.60:0/1914825394 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f08d00075e0 con 0x7f08e0096860 2023-08-24T10:07:20.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.820+0000 7f08e499e640 1 -- 172.21.15.60:0/1914825394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e01c4e70 con 0x7f08e0096860 2023-08-24T10:07:20.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.821+0000 7f08e499e640 1 -- 172.21.15.60:0/1914825394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e0061800 con 0x7f08e0096860 2023-08-24T10:07:20.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.821+0000 7f08d77fe640 1 -- 172.21.15.60:0/1914825394 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f08d00057e0 con 0x7f08e0096860 2023-08-24T10:07:20.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.822+0000 7f08d77fe640 1 --2- 172.21.15.60:0/1914825394 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f08c0037130 0x7f08c0039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:20.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.822+0000 7f08d77fe640 1 -- 172.21.15.60:0/1914825394 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f08d0046340 con 0x7f08e0096860 2023-08-24T10:07:20.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.822+0000 7f08d5d9b640 1 --2- 172.21.15.60:0/1914825394 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f08c0037130 0x7f08c0039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:20.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.822+0000 7f08d5d9b640 1 --2- 172.21.15.60:0/1914825394 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f08c0037130 0x7f08c0039640 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:20.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.826+0000 7f08d77fe640 1 -- 172.21.15.60:0/1914825394 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f08d000d330 con 0x7f08e0096860 2023-08-24T10:07:21.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:20.999+0000 7f08e499e640 1 -- 172.21.15.60:0/1914825394 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f08e006d190 con 0x7f08c0037130 2023-08-24T10:07:21.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.001+0000 7f08d77fe640 1 -- 172.21.15.60:0/1914825394 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f08c8009df0 con 0x7f08c0037130 2023-08-24T10:07:21.002 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:21.002 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:21.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.002+0000 7f08e499e640 1 -- 172.21.15.60:0/1914825394 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f08c0037130 msgr2=0x7f08c0039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:21.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.002+0000 7f08e499e640 1 --2- 172.21.15.60:0/1914825394 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f08c0037130 0x7f08c0039640 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:21.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.002+0000 7f08e499e640 1 -- 172.21.15.60:0/1914825394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08e0096860 msgr2=0x7f08e01c52a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.002+0000 7f08e499e640 1 --2- 172.21.15.60:0/1914825394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08e0096860 0x7f08e01c52a0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.002+0000 7f08e499e640 1 -- 172.21.15.60:0/1914825394 shutdown_connections 2023-08-24T10:07:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.002+0000 7f08e499e640 1 --2- 172.21.15.60:0/1914825394 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f08c0037130 0x7f08c0039640 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.002+0000 7f08e499e640 1 --2- 172.21.15.60:0/1914825394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08e0096860 0x7f08e01c52a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.002+0000 7f08e499e640 1 -- 172.21.15.60:0/1914825394 >> 172.21.15.60:0/1914825394 conn(0x7f08e009e110 msgr2=0x7f08e00950c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.002+0000 7f08e499e640 1 -- 172.21.15.60:0/1914825394 shutdown_connections 2023-08-24T10:07:21.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.002+0000 7f08e499e640 1 -- 172.21.15.60:0/1914825394 wait complete. 2023-08-24T10:07:21.014 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2023-08-24T10:07:19.523065+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:21.016 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:07:21.016 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:21.017 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:07:21.017 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:21.018 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:07:21.018 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:21.018 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:07:21.018 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:21.019 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:07:21.019 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:21.019 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:07:21.019 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:21.020 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:07:21.021 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:21.021 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:07:21.022 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:21.022 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:07:21.023 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:21.024 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:07:21.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.131+0000 7f86774ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:21.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.134+0000 7f86774ac640 1 Processor -- start 2023-08-24T10:07:21.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.134+0000 7f86774ac640 1 -- start start 2023-08-24T10:07:21.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.135+0000 7f86774ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f867012ece0 0x7f867012f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:21.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.135+0000 7f86774ac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867009a650 con 0x7f867012f630 2023-08-24T10:07:21.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.135+0000 7f86774ac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867006da00 con 0x7f867012ece0 2023-08-24T10:07:21.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.135+0000 7f8675248640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f867012ece0 0x7f867012f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:21.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.135+0000 7f8675248640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f867012ece0 0x7f867012f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39252/0 (socket says 172.21.15.60:39252) 2023-08-24T10:07:21.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.135+0000 7f8674a47640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f867012f630 0x7f8670097fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56472/0 (socket says 172.21.15.60:56472) 2023-08-24T10:07:21.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.135+0000 7f8675248640 1 -- 172.21.15.60:0/100909903 learned_addr learned my addr 172.21.15.60:0/100909903 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:21.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.135+0000 7f8675248640 1 -- 172.21.15.60:0/100909903 >> v1:172.21.15.60:6789/0 conn(0x7f867012f630 legacy=0x7f8670097fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:21.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.135+0000 7f8675248640 1 -- 172.21.15.60:0/100909903 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867012ea10 con 0x7f867012ece0 2023-08-24T10:07:21.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.136+0000 7f8675248640 1 --2- 172.21.15.60:0/100909903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f867012ece0 0x7f867012f0f0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7ba4e0571f477c server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:21.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.136+0000 7f866ffff640 1 -- 172.21.15.60:0/100909903 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f866800ad60 con 0x7f867012ece0 2023-08-24T10:07:21.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.136+0000 7f866ffff640 1 -- 172.21.15.60:0/100909903 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f866800f340 con 0x7f867012ece0 2023-08-24T10:07:21.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.136+0000 7f866ffff640 1 -- 172.21.15.60:0/100909903 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f866800f5e0 con 0x7f867012ece0 2023-08-24T10:07:21.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.136+0000 7f86774ac640 1 -- 172.21.15.60:0/100909903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f867012ece0 msgr2=0x7f867012f0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:21.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.136+0000 7f86774ac640 1 --2- 172.21.15.60:0/100909903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f867012ece0 0x7f867012f0f0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:21.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.136+0000 7f86774ac640 1 -- 172.21.15.60:0/100909903 shutdown_connections 2023-08-24T10:07:21.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.136+0000 7f86774ac640 1 --2- 172.21.15.60:0/100909903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f867012ece0 0x7f867012f0f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:21.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.136+0000 7f86774ac640 1 -- 172.21.15.60:0/100909903 >> 172.21.15.60:0/100909903 conn(0x7f867009e110 msgr2=0x7f867009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:21.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.137+0000 7f86774ac640 1 -- 172.21.15.60:0/100909903 shutdown_connections 2023-08-24T10:07:21.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.137+0000 7f86774ac640 1 -- 172.21.15.60:0/100909903 wait complete. 2023-08-24T10:07:21.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.137+0000 7f86774ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:21.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.138+0000 7f86774ac640 1 Processor -- start 2023-08-24T10:07:21.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.138+0000 7f86774ac640 1 -- start start 2023-08-24T10:07:21.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.138+0000 7f86774ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f867012ece0 0x7f867012b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:21.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.138+0000 7f86774ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867012b140 con 0x7f867012ece0 2023-08-24T10:07:21.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.138+0000 7f8675248640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f867012ece0 0x7f867012b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:21.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.138+0000 7f8675248640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f867012ece0 0x7f867012b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39256/0 (socket says 172.21.15.60:39256) 2023-08-24T10:07:21.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.138+0000 7f8675248640 1 -- 172.21.15.60:0/4179250926 learned_addr learned my addr 172.21.15.60:0/4179250926 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:21.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.139+0000 7f8675248640 1 -- 172.21.15.60:0/4179250926 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8670089380 con 0x7f867012ece0 2023-08-24T10:07:21.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.139+0000 7f8675248640 1 --2- 172.21.15.60:0/4179250926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f867012ece0 0x7f867012b500 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:21.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.139+0000 7f866dffb640 1 -- 172.21.15.60:0/4179250926 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f866800fd70 con 0x7f867012ece0 2023-08-24T10:07:21.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.139+0000 7f866dffb640 1 -- 172.21.15.60:0/4179250926 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f866800fed0 con 0x7f867012ece0 2023-08-24T10:07:21.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.139+0000 7f86774ac640 1 -- 172.21.15.60:0/4179250926 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867008b810 con 0x7f867012ece0 2023-08-24T10:07:21.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.139+0000 7f866dffb640 1 -- 172.21.15.60:0/4179250926 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8668011550 con 0x7f867012ece0 2023-08-24T10:07:21.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.139+0000 7f86774ac640 1 -- 172.21.15.60:0/4179250926 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867008a110 con 0x7f867012ece0 2023-08-24T10:07:21.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.140+0000 7f866dffb640 1 -- 172.21.15.60:0/4179250926 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f8668011700 con 0x7f867012ece0 2023-08-24T10:07:21.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.140+0000 7f866dffb640 1 --2- 172.21.15.60:0/4179250926 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f865c037130 0x7f865c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:21.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.140+0000 7f866dffb640 1 -- 172.21.15.60:0/4179250926 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f8668017040 con 0x7f867012ece0 2023-08-24T10:07:21.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.140+0000 7f86774ac640 1 -- 172.21.15.60:0/4179250926 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8670096e80 con 0x7f867012ece0 2023-08-24T10:07:21.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.140+0000 7f8674a47640 1 --2- 172.21.15.60:0/4179250926 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f865c037130 0x7f865c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:21.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.141+0000 7f8674a47640 1 --2- 172.21.15.60:0/4179250926 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f865c037130 0x7f865c039640 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:21.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.145+0000 7f866dffb640 1 -- 172.21.15.60:0/4179250926 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f86680103b0 con 0x7f867012ece0 2023-08-24T10:07:21.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.387+0000 7f86774ac640 1 -- 172.21.15.60:0/4179250926 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8670061800 con 0x7f867012ece0 2023-08-24T10:07:21.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.387+0000 7f866dffb640 1 -- 172.21.15.60:0/4179250926 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f866800b3c0 con 0x7f867012ece0 2023-08-24T10:07:21.388 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:21.389 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":152,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:07:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.389+0000 7f86774ac640 1 -- 172.21.15.60:0/4179250926 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f865c037130 msgr2=0x7f865c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.389+0000 7f86774ac640 1 --2- 172.21.15.60:0/4179250926 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f865c037130 0x7f865c039640 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:21.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.389+0000 7f86774ac640 1 -- 172.21.15.60:0/4179250926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f867012ece0 msgr2=0x7f867012b500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.389+0000 7f86774ac640 1 --2- 172.21.15.60:0/4179250926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f867012ece0 0x7f867012b500 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.389+0000 7f86774ac640 1 -- 172.21.15.60:0/4179250926 shutdown_connections 2023-08-24T10:07:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.389+0000 7f86774ac640 1 --2- 172.21.15.60:0/4179250926 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f865c037130 0x7f865c039640 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.389+0000 7f86774ac640 1 --2- 172.21.15.60:0/4179250926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f867012ece0 0x7f867012b500 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.389+0000 7f86774ac640 1 -- 172.21.15.60:0/4179250926 >> 172.21.15.60:0/4179250926 conn(0x7f867009e110 msgr2=0x7f8670097710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.389+0000 7f86774ac640 1 -- 172.21.15.60:0/4179250926 shutdown_connections 2023-08-24T10:07:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.389+0000 7f86774ac640 1 -- 172.21.15.60:0/4179250926 wait complete. 2023-08-24T10:07:21.401 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:07:21.401 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:21.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.508+0000 7f17148ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:21.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.512+0000 7f17148ec640 1 Processor -- start 2023-08-24T10:07:21.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.512+0000 7f17148ec640 1 -- start start 2023-08-24T10:07:21.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.513+0000 7f17148ec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1710135980 0x7f1710137da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:21.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.513+0000 7f17148ec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1710132610 con 0x7f1710132b50 2023-08-24T10:07:21.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.513+0000 7f17148ec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f171009c260 con 0x7f1710135980 2023-08-24T10:07:21.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.513+0000 7f170dd9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1710135980 0x7f1710137da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:21.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.513+0000 7f170dd9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1710135980 0x7f1710137da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39270/0 (socket says 172.21.15.60:39270) 2023-08-24T10:07:21.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.513+0000 7f170dd9b640 1 -- 172.21.15.60:0/2241009499 learned_addr learned my addr 172.21.15.60:0/2241009499 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:21.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.514+0000 7f170dd9b640 1 -- 172.21.15.60:0/2241009499 >> v1:172.21.15.60:6789/0 conn(0x7f1710132b50 legacy=0x7f1710132f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:21.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.514+0000 7f170dd9b640 1 -- 172.21.15.60:0/2241009499 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1710132840 con 0x7f1710135980 2023-08-24T10:07:21.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.514+0000 7f170dd9b640 1 --2- 172.21.15.60:0/2241009499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1710135980 0x7f1710137da0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f886432e8b57eed server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:21.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.514+0000 7f170d59a640 1 -- 172.21.15.60:0/2241009499 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1704009dd0 con 0x7f1710135980 2023-08-24T10:07:21.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.514+0000 7f170d59a640 1 -- 172.21.15.60:0/2241009499 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f170400e230 con 0x7f1710135980 2023-08-24T10:07:21.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.514+0000 7f170d59a640 1 -- 172.21.15.60:0/2241009499 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f170400e4d0 con 0x7f1710135980 2023-08-24T10:07:21.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.514+0000 7f17148ec640 1 -- 172.21.15.60:0/2241009499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1710135980 msgr2=0x7f1710137da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:21.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.514+0000 7f17148ec640 1 --2- 172.21.15.60:0/2241009499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1710135980 0x7f1710137da0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.515+0000 7f17148ec640 1 -- 172.21.15.60:0/2241009499 shutdown_connections 2023-08-24T10:07:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.515+0000 7f17148ec640 1 --2- 172.21.15.60:0/2241009499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1710135980 0x7f1710137da0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.515+0000 7f17148ec640 1 -- 172.21.15.60:0/2241009499 >> 172.21.15.60:0/2241009499 conn(0x7f17100a26a0 msgr2=0x7f17100a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.515+0000 7f17148ec640 1 -- 172.21.15.60:0/2241009499 shutdown_connections 2023-08-24T10:07:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.515+0000 7f17148ec640 1 -- 172.21.15.60:0/2241009499 wait complete. 2023-08-24T10:07:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.515+0000 7f17148ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:21.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.516+0000 7f17148ec640 1 Processor -- start 2023-08-24T10:07:21.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.516+0000 7f17148ec640 1 -- start start 2023-08-24T10:07:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.517+0000 7f17148ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1710132b50 0x7f17100a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.517+0000 7f17148ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17100a1b60 con 0x7f1710132b50 2023-08-24T10:07:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.517+0000 7f170e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1710132b50 0x7f17100a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.517+0000 7f170e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1710132b50 0x7f17100a0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39276/0 (socket says 172.21.15.60:39276) 2023-08-24T10:07:21.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.517+0000 7f170e59c640 1 -- 172.21.15.60:0/2576862612 learned_addr learned my addr 172.21.15.60:0/2576862612 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.517+0000 7f170e59c640 1 -- 172.21.15.60:0/2576862612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171009fab0 con 0x7f1710132b50 2023-08-24T10:07:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.517+0000 7f170e59c640 1 --2- 172.21.15.60:0/2576862612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1710132b50 0x7f17100a0000 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.517+0000 7f16feffd640 1 -- 172.21.15.60:0/2576862612 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f170000c860 con 0x7f1710132b50 2023-08-24T10:07:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.517+0000 7f16feffd640 1 -- 172.21.15.60:0/2576862612 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f170000c9c0 con 0x7f1710132b50 2023-08-24T10:07:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.517+0000 7f17148ec640 1 -- 172.21.15.60:0/2576862612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171006a240 con 0x7f1710132b50 2023-08-24T10:07:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.517+0000 7f16feffd640 1 -- 172.21.15.60:0/2576862612 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f170000cc60 con 0x7f1710132b50 2023-08-24T10:07:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.517+0000 7f17148ec640 1 -- 172.21.15.60:0/2576862612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171011ac50 con 0x7f1710132b50 2023-08-24T10:07:21.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.518+0000 7f17148ec640 1 -- 172.21.15.60:0/2576862612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1710061800 con 0x7f1710132b50 2023-08-24T10:07:21.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.518+0000 7f16feffd640 1 -- 172.21.15.60:0/2576862612 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f1700011310 con 0x7f1710132b50 2023-08-24T10:07:21.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.518+0000 7f16feffd640 1 --2- 172.21.15.60:0/2576862612 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f16f0037130 0x7f16f0039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:21.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.519+0000 7f16feffd640 1 -- 172.21.15.60:0/2576862612 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f1700046910 con 0x7f1710132b50 2023-08-24T10:07:21.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.519+0000 7f170dd9b640 1 --2- 172.21.15.60:0/2576862612 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f16f0037130 0x7f16f0039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:21.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.519+0000 7f170dd9b640 1 --2- 172.21.15.60:0/2576862612 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f16f0037130 0x7f16f0039640 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:21.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.523+0000 7f16feffd640 1 -- 172.21.15.60:0/2576862612 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f170000a590 con 0x7f1710132b50 2023-08-24T10:07:21.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.695+0000 7f17148ec640 1 -- 172.21.15.60:0/2576862612 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f171009f940 con 0x7f16f0037130 2023-08-24T10:07:21.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.696+0000 7f16feffd640 1 -- 172.21.15.60:0/2576862612 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f171009f940 con 0x7f16f0037130 2023-08-24T10:07:21.698 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:21.700 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:21.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.698+0000 7f17148ec640 1 -- 172.21.15.60:0/2576862612 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f16f0037130 msgr2=0x7f16f0039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:21.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.698+0000 7f17148ec640 1 --2- 172.21.15.60:0/2576862612 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f16f0037130 0x7f16f0039640 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:21.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.698+0000 7f17148ec640 1 -- 172.21.15.60:0/2576862612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1710132b50 msgr2=0x7f17100a0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:21.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.698+0000 7f17148ec640 1 --2- 172.21.15.60:0/2576862612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1710132b50 0x7f17100a0000 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:21.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.698+0000 7f17148ec640 1 -- 172.21.15.60:0/2576862612 shutdown_connections 2023-08-24T10:07:21.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.698+0000 7f17148ec640 1 --2- 172.21.15.60:0/2576862612 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f16f0037130 0x7f16f0039640 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:21.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.698+0000 7f17148ec640 1 --2- 172.21.15.60:0/2576862612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1710132b50 0x7f17100a0000 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:21.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.698+0000 7f17148ec640 1 -- 172.21.15.60:0/2576862612 >> 172.21.15.60:0/2576862612 conn(0x7f17100a26a0 msgr2=0x7f17101baaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:21.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.698+0000 7f17148ec640 1 -- 172.21.15.60:0/2576862612 shutdown_connections 2023-08-24T10:07:21.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:21.698+0000 7f17148ec640 1 -- 172.21.15.60:0/2576862612 wait complete. 2023-08-24T10:07:21.710 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2023-08-24T10:07:21.523404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:22.746 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~135s 2023-08-24T10:07:22.747 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~135s 2023-08-24T10:07:22.747 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~135s 2023-08-24T10:07:22.747 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~135s 2023-08-24T10:07:24.711 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:24.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.832+0000 7f58652ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:24.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.836+0000 7f58652ec640 1 Processor -- start 2023-08-24T10:07:24.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.836+0000 7f58652ec640 1 -- start start 2023-08-24T10:07:24.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.837+0000 7f58652ec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5860132630 0x7f5860134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:24.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.837+0000 7f58652ec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f586006e320 con 0x7f5860096860 2023-08-24T10:07:24.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.837+0000 7f58652ec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f586006db70 con 0x7f5860132630 2023-08-24T10:07:24.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.837+0000 7f585e7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5860132630 0x7f5860134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:24.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.837+0000 7f585e7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5860132630 0x7f5860134a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39284/0 (socket says 172.21.15.60:39284) 2023-08-24T10:07:24.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.837+0000 7f585e7fc640 1 -- 172.21.15.60:0/1782665523 learned_addr learned my addr 172.21.15.60:0/1782665523 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:24.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.837+0000 7f585effd640 1 --1- 172.21.15.60:0/1782665523 >> v1:172.21.15.60:6789/0 conn(0x7f5860096860 0x7f5860096c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56490/0 (socket says 172.21.15.60:56490) 2023-08-24T10:07:24.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.837+0000 7f585e7fc640 1 -- 172.21.15.60:0/1782665523 >> v1:172.21.15.60:6789/0 conn(0x7f5860096860 legacy=0x7f5860096c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:24.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.837+0000 7f585e7fc640 1 -- 172.21.15.60:0/1782665523 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f586006cef0 con 0x7f5860132630 2023-08-24T10:07:24.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.837+0000 7f585e7fc640 1 --2- 172.21.15.60:0/1782665523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5860132630 0x7f5860134a50 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6225272261c562be server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:24.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.838+0000 7f585dffb640 1 -- 172.21.15.60:0/1782665523 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5854009dd0 con 0x7f5860132630 2023-08-24T10:07:24.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.838+0000 7f585dffb640 1 -- 172.21.15.60:0/1782665523 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f585400e230 con 0x7f5860132630 2023-08-24T10:07:24.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.838+0000 7f585dffb640 1 -- 172.21.15.60:0/1782665523 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f585400e4d0 con 0x7f5860132630 2023-08-24T10:07:24.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.838+0000 7f58652ec640 1 -- 172.21.15.60:0/1782665523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5860132630 msgr2=0x7f5860134a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:24.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.838+0000 7f58652ec640 1 --2- 172.21.15.60:0/1782665523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5860132630 0x7f5860134a50 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:24.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.838+0000 7f58652ec640 1 -- 172.21.15.60:0/1782665523 shutdown_connections 2023-08-24T10:07:24.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.838+0000 7f58652ec640 1 --2- 172.21.15.60:0/1782665523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5860132630 0x7f5860134a50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:24.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.838+0000 7f58652ec640 1 -- 172.21.15.60:0/1782665523 >> 172.21.15.60:0/1782665523 conn(0x7f586009e110 msgr2=0x7f586009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:24.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.838+0000 7f58652ec640 1 -- 172.21.15.60:0/1782665523 shutdown_connections 2023-08-24T10:07:24.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.838+0000 7f58652ec640 1 -- 172.21.15.60:0/1782665523 wait complete. 2023-08-24T10:07:24.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.839+0000 7f58652ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:24.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.840+0000 7f58652ec640 1 Processor -- start 2023-08-24T10:07:24.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.840+0000 7f58652ec640 1 -- start start 2023-08-24T10:07:24.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.840+0000 7f58652ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5860096860 0x7f5860093330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:24.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.841+0000 7f58652ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58600905a0 con 0x7f5860096860 2023-08-24T10:07:24.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.841+0000 7f585effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5860096860 0x7f5860093330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:24.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.841+0000 7f585effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5860096860 0x7f5860093330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39294/0 (socket says 172.21.15.60:39294) 2023-08-24T10:07:24.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.841+0000 7f585effd640 1 -- 172.21.15.60:0/2778667549 learned_addr learned my addr 172.21.15.60:0/2778667549 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:24.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.841+0000 7f585effd640 1 -- 172.21.15.60:0/2778667549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5860092b70 con 0x7f5860096860 2023-08-24T10:07:24.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.841+0000 7f585effd640 1 --2- 172.21.15.60:0/2778667549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5860096860 0x7f5860093330 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:24.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.841+0000 7f58477fe640 1 -- 172.21.15.60:0/2778667549 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f58500071e0 con 0x7f5860096860 2023-08-24T10:07:24.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.841+0000 7f58477fe640 1 -- 172.21.15.60:0/2778667549 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5850007340 con 0x7f5860096860 2023-08-24T10:07:24.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.841+0000 7f58652ec640 1 -- 172.21.15.60:0/2778667549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5860092f90 con 0x7f5860096860 2023-08-24T10:07:24.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.841+0000 7f58477fe640 1 -- 172.21.15.60:0/2778667549 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f58500075e0 con 0x7f5860096860 2023-08-24T10:07:24.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.841+0000 7f58652ec640 1 -- 172.21.15.60:0/2778667549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5860091430 con 0x7f5860096860 2023-08-24T10:07:24.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.842+0000 7f58477fe640 1 -- 172.21.15.60:0/2778667549 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f58500057e0 con 0x7f5860096860 2023-08-24T10:07:24.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.842+0000 7f58477fe640 1 --2- 172.21.15.60:0/2778667549 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f584c037130 0x7f584c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:24.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.842+0000 7f58477fe640 1 -- 172.21.15.60:0/2778667549 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f5850046430 con 0x7f5860096860 2023-08-24T10:07:24.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.842+0000 7f58652ec640 1 -- 172.21.15.60:0/2778667549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5860090e80 con 0x7f5860096860 2023-08-24T10:07:24.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.843+0000 7f585e7fc640 1 --2- 172.21.15.60:0/2778667549 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f584c037130 0x7f584c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:24.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.843+0000 7f585e7fc640 1 --2- 172.21.15.60:0/2778667549 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f584c037130 0x7f584c039640 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:24.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:24.847+0000 7f58477fe640 1 -- 172.21.15.60:0/2778667549 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f585000d330 con 0x7f5860096860 2023-08-24T10:07:25.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.022+0000 7f58652ec640 1 -- 172.21.15.60:0/2778667549 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5860092d70 con 0x7f584c037130 2023-08-24T10:07:25.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.023+0000 7f58477fe640 1 -- 172.21.15.60:0/2778667549 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f5854009df0 con 0x7f584c037130 2023-08-24T10:07:25.024 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:25.025 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:25.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.025+0000 7f58652ec640 1 -- 172.21.15.60:0/2778667549 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f584c037130 msgr2=0x7f584c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:25.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.025+0000 7f58652ec640 1 --2- 172.21.15.60:0/2778667549 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f584c037130 0x7f584c039640 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:25.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.025+0000 7f58652ec640 1 -- 172.21.15.60:0/2778667549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5860096860 msgr2=0x7f5860093330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:25.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.025+0000 7f58652ec640 1 --2- 172.21.15.60:0/2778667549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5860096860 0x7f5860093330 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:25.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.025+0000 7f58652ec640 1 -- 172.21.15.60:0/2778667549 shutdown_connections 2023-08-24T10:07:25.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.025+0000 7f58652ec640 1 --2- 172.21.15.60:0/2778667549 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f584c037130 0x7f584c039640 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:25.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.025+0000 7f58652ec640 1 --2- 172.21.15.60:0/2778667549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5860096860 0x7f5860093330 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:25.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.025+0000 7f58652ec640 1 -- 172.21.15.60:0/2778667549 >> 172.21.15.60:0/2778667549 conn(0x7f586009e110 msgr2=0x7f5860097fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:25.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.025+0000 7f58652ec640 1 -- 172.21.15.60:0/2778667549 shutdown_connections 2023-08-24T10:07:25.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.025+0000 7f58652ec640 1 -- 172.21.15.60:0/2778667549 wait complete. 2023-08-24T10:07:25.038 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2023-08-24T10:07:23.523809+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:25.039 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:25.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.146+0000 7f887039e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:25.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.150+0000 7f887039e640 1 Processor -- start 2023-08-24T10:07:25.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.150+0000 7f887039e640 1 -- start start 2023-08-24T10:07:25.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.150+0000 7f887039e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8868096860 0x7f8868096c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:25.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.150+0000 7f887039e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88680971b0 con 0x7f88680990b0 2023-08-24T10:07:25.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.150+0000 7f887039e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f886806db70 con 0x7f8868096860 2023-08-24T10:07:25.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.151+0000 7f886e13a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8868096860 0x7f8868096c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:25.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.151+0000 7f886e13a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8868096860 0x7f8868096c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39304/0 (socket says 172.21.15.60:39304) 2023-08-24T10:07:25.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.151+0000 7f886e13a640 1 -- 172.21.15.60:0/1820105425 learned_addr learned my addr 172.21.15.60:0/1820105425 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:25.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.151+0000 7f886d939640 1 --1- 172.21.15.60:0/1820105425 >> v1:172.21.15.60:6789/0 conn(0x7f88680990b0 0x7f88680994c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56492/0 (socket says 172.21.15.60:56492) 2023-08-24T10:07:25.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.151+0000 7f886e13a640 1 -- 172.21.15.60:0/1820105425 >> v1:172.21.15.60:6789/0 conn(0x7f88680990b0 legacy=0x7f88680994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:25.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.151+0000 7f886e13a640 1 -- 172.21.15.60:0/1820105425 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886806e320 con 0x7f8868096860 2023-08-24T10:07:25.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.151+0000 7f886e13a640 1 --2- 172.21.15.60:0/1820105425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8868096860 0x7f8868096c70 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2617950b15c846 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:25.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.151+0000 7f886d138640 1 -- 172.21.15.60:0/1820105425 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8864009dd0 con 0x7f8868096860 2023-08-24T10:07:25.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.151+0000 7f886d138640 1 -- 172.21.15.60:0/1820105425 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f886400e230 con 0x7f8868096860 2023-08-24T10:07:25.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.151+0000 7f886d138640 1 -- 172.21.15.60:0/1820105425 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f886400e4d0 con 0x7f8868096860 2023-08-24T10:07:25.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.152+0000 7f887039e640 1 -- 172.21.15.60:0/1820105425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8868096860 msgr2=0x7f8868096c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:25.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.152+0000 7f887039e640 1 --2- 172.21.15.60:0/1820105425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8868096860 0x7f8868096c70 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:25.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.152+0000 7f887039e640 1 -- 172.21.15.60:0/1820105425 shutdown_connections 2023-08-24T10:07:25.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.152+0000 7f887039e640 1 --2- 172.21.15.60:0/1820105425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8868096860 0x7f8868096c70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:25.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.152+0000 7f887039e640 1 -- 172.21.15.60:0/1820105425 >> 172.21.15.60:0/1820105425 conn(0x7f886809e110 msgr2=0x7f886809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:25.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.152+0000 7f887039e640 1 -- 172.21.15.60:0/1820105425 shutdown_connections 2023-08-24T10:07:25.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.152+0000 7f887039e640 1 -- 172.21.15.60:0/1820105425 wait complete. 2023-08-24T10:07:25.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.152+0000 7f887039e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:25.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.153+0000 7f887039e640 1 Processor -- start 2023-08-24T10:07:25.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.153+0000 7f887039e640 1 -- start start 2023-08-24T10:07:25.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.154+0000 7f887039e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8868096860 0x7f886812af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:25.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.154+0000 7f887039e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8868129890 con 0x7f8868096860 2023-08-24T10:07:25.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.154+0000 7f886e13a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8868096860 0x7f886812af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:25.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.154+0000 7f886e13a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8868096860 0x7f886812af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39308/0 (socket says 172.21.15.60:39308) 2023-08-24T10:07:25.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.154+0000 7f886e13a640 1 -- 172.21.15.60:0/22801306 learned_addr learned my addr 172.21.15.60:0/22801306 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:25.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.154+0000 7f886e13a640 1 -- 172.21.15.60:0/22801306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88681bf650 con 0x7f8868096860 2023-08-24T10:07:25.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.154+0000 7f886e13a640 1 --2- 172.21.15.60:0/22801306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8868096860 0x7f886812af10 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:25.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.154+0000 7f885effd640 1 -- 172.21.15.60:0/22801306 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8864014540 con 0x7f8868096860 2023-08-24T10:07:25.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.154+0000 7f885effd640 1 -- 172.21.15.60:0/22801306 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f88640146a0 con 0x7f8868096860 2023-08-24T10:07:25.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.154+0000 7f885effd640 1 -- 172.21.15.60:0/22801306 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8864014940 con 0x7f8868096860 2023-08-24T10:07:25.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.154+0000 7f887039e640 1 -- 172.21.15.60:0/22801306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88681bf960 con 0x7f8868096860 2023-08-24T10:07:25.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.154+0000 7f887039e640 1 -- 172.21.15.60:0/22801306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f886812a940 con 0x7f8868096860 2023-08-24T10:07:25.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.155+0000 7f885effd640 1 -- 172.21.15.60:0/22801306 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f8864014af0 con 0x7f8868096860 2023-08-24T10:07:25.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.155+0000 7f887039e640 1 -- 172.21.15.60:0/22801306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8868061800 con 0x7f8868096860 2023-08-24T10:07:25.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.155+0000 7f885effd640 1 --2- 172.21.15.60:0/22801306 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f884c037130 0x7f884c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:25.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.156+0000 7f885effd640 1 -- 172.21.15.60:0/22801306 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f8864019040 con 0x7f8868096860 2023-08-24T10:07:25.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.156+0000 7f886d939640 1 --2- 172.21.15.60:0/22801306 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f884c037130 0x7f884c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:25.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.156+0000 7f886d939640 1 --2- 172.21.15.60:0/22801306 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f884c037130 0x7f884c039640 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:25.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.160+0000 7f885effd640 1 -- 172.21.15.60:0/22801306 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f8864018190 con 0x7f8868096860 2023-08-24T10:07:25.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.333+0000 7f887039e640 1 -- 172.21.15.60:0/22801306 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f886812bf20 con 0x7f884c037130 2023-08-24T10:07:25.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.335+0000 7f885effd640 1 -- 172.21.15.60:0/22801306 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f886812bf20 con 0x7f884c037130 2023-08-24T10:07:25.337 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:25.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.337+0000 7f887039e640 1 -- 172.21.15.60:0/22801306 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f884c037130 msgr2=0x7f884c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:25.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.337+0000 7f887039e640 1 --2- 172.21.15.60:0/22801306 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f884c037130 0x7f884c039640 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:25.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.337+0000 7f887039e640 1 -- 172.21.15.60:0/22801306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8868096860 msgr2=0x7f886812af10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:25.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.337+0000 7f887039e640 1 --2- 172.21.15.60:0/22801306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8868096860 0x7f886812af10 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:25.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.337+0000 7f887039e640 1 -- 172.21.15.60:0/22801306 shutdown_connections 2023-08-24T10:07:25.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.337+0000 7f887039e640 1 --2- 172.21.15.60:0/22801306 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f884c037130 0x7f884c039640 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:25.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.337+0000 7f887039e640 1 --2- 172.21.15.60:0/22801306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8868096860 0x7f886812af10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:25.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.337+0000 7f887039e640 1 -- 172.21.15.60:0/22801306 >> 172.21.15.60:0/22801306 conn(0x7f886809e110 msgr2=0x7f88681b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:25.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.337+0000 7f887039e640 1 -- 172.21.15.60:0/22801306 shutdown_connections 2023-08-24T10:07:25.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.337+0000 7f887039e640 1 -- 172.21.15.60:0/22801306 wait complete. 2023-08-24T10:07:25.349 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2023-08-24T10:07:23.523809+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:25.350 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:07:25.350 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:25.351 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:07:25.351 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:25.351 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:07:25.351 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:25.352 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:07:25.352 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:25.352 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:07:25.352 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:25.353 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:07:25.353 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:25.353 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:07:25.353 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:25.354 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:07:25.354 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:25.354 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:07:25.355 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:25.355 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:07:25.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.464+0000 7fc70e71e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:25.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.467+0000 7fc70e71e640 1 Processor -- start 2023-08-24T10:07:25.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.467+0000 7fc70e71e640 1 -- start start 2023-08-24T10:07:25.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.468+0000 7fc70e71e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc708131d20 0x7fc708134140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:25.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.468+0000 7fc70e71e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc70809a650 con 0x7fc70812ece0 2023-08-24T10:07:25.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.468+0000 7fc70e71e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc70806da00 con 0x7fc708131d20 2023-08-24T10:07:25.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.468+0000 7fc707fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc70812ece0 0x7fc70812f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56502/0 (socket says 172.21.15.60:56502) 2023-08-24T10:07:25.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.468+0000 7fc707fff640 1 -- 172.21.15.60:0/984813221 learned_addr learned my addr 172.21.15.60:0/984813221 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:07:25.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.468+0000 7fc7077fe640 1 --2- 172.21.15.60:0/984813221 >> v2:172.21.15.60:3300/0 conn(0x7fc708131d20 0x7fc708134140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:25.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.468+0000 7fc7077fe640 1 -- 172.21.15.60:0/984813221 >> v1:172.21.15.60:6789/0 conn(0x7fc70812ece0 legacy=0x7fc70812f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:25.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.468+0000 7fc7077fe640 1 -- 172.21.15.60:0/984813221 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70812ea10 con 0x7fc708131d20 2023-08-24T10:07:25.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.468+0000 7fc7077fe640 1 --2- 172.21.15.60:0/984813221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc708131d20 0x7fc708134140 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73ae22cb8edc7 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:25.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.468+0000 7fc706ffd640 1 -- 172.21.15.60:0/984813221 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc6fc009ce0 con 0x7fc708131d20 2023-08-24T10:07:25.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.468+0000 7fc706ffd640 1 -- 172.21.15.60:0/984813221 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc6fc009e40 con 0x7fc708131d20 2023-08-24T10:07:25.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.468+0000 7fc706ffd640 1 -- 172.21.15.60:0/984813221 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc6fc00f470 con 0x7fc708131d20 2023-08-24T10:07:25.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.469+0000 7fc70e71e640 1 -- 172.21.15.60:0/984813221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc708131d20 msgr2=0x7fc708134140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:25.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.469+0000 7fc70e71e640 1 --2- 172.21.15.60:0/984813221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc708131d20 0x7fc708134140 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:25.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.469+0000 7fc70e71e640 1 -- 172.21.15.60:0/984813221 shutdown_connections 2023-08-24T10:07:25.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.469+0000 7fc70e71e640 1 --2- 172.21.15.60:0/984813221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc708131d20 0x7fc708134140 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:25.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.469+0000 7fc70e71e640 1 -- 172.21.15.60:0/984813221 >> 172.21.15.60:0/984813221 conn(0x7fc70809e110 msgr2=0x7fc70809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:25.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.469+0000 7fc70e71e640 1 -- 172.21.15.60:0/984813221 shutdown_connections 2023-08-24T10:07:25.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.469+0000 7fc70e71e640 1 -- 172.21.15.60:0/984813221 wait complete. 2023-08-24T10:07:25.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.469+0000 7fc70e71e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:25.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.470+0000 7fc70e71e640 1 Processor -- start 2023-08-24T10:07:25.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.470+0000 7fc70e71e640 1 -- start start 2023-08-24T10:07:25.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.471+0000 7fc70e71e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc70812ece0 0x7fc708070060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:25.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.471+0000 7fc70e71e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc708071450 con 0x7fc70812ece0 2023-08-24T10:07:25.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.471+0000 7fc707fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc70812ece0 0x7fc708070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:25.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.471+0000 7fc707fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc70812ece0 0x7fc708070060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39328/0 (socket says 172.21.15.60:39328) 2023-08-24T10:07:25.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.471+0000 7fc707fff640 1 -- 172.21.15.60:0/714605844 learned_addr learned my addr 172.21.15.60:0/714605844 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:25.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.471+0000 7fc707fff640 1 -- 172.21.15.60:0/714605844 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc708129610 con 0x7fc70812ece0 2023-08-24T10:07:25.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.472+0000 7fc707fff640 1 --2- 172.21.15.60:0/714605844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc70812ece0 0x7fc708070060 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:25.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.472+0000 7fc704ff9640 1 -- 172.21.15.60:0/714605844 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc6f80059d0 con 0x7fc70812ece0 2023-08-24T10:07:25.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.472+0000 7fc704ff9640 1 -- 172.21.15.60:0/714605844 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc6f8005b30 con 0x7fc70812ece0 2023-08-24T10:07:25.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.472+0000 7fc70e71e640 1 -- 172.21.15.60:0/714605844 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc708128a20 con 0x7fc70812ece0 2023-08-24T10:07:25.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.472+0000 7fc704ff9640 1 -- 172.21.15.60:0/714605844 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc6f8005dd0 con 0x7fc70812ece0 2023-08-24T10:07:25.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.472+0000 7fc70e71e640 1 -- 172.21.15.60:0/714605844 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc708127990 con 0x7fc70812ece0 2023-08-24T10:07:25.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.473+0000 7fc704ff9640 1 -- 172.21.15.60:0/714605844 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fc6f80070b0 con 0x7fc70812ece0 2023-08-24T10:07:25.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.473+0000 7fc704ff9640 1 --2- 172.21.15.60:0/714605844 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc6f4037130 0x7fc6f4039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:25.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.473+0000 7fc704ff9640 1 -- 172.21.15.60:0/714605844 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fc6f8046340 con 0x7fc70812ece0 2023-08-24T10:07:25.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.473+0000 7fc70e71e640 1 -- 172.21.15.60:0/714605844 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc708061800 con 0x7fc70812ece0 2023-08-24T10:07:25.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.473+0000 7fc7077fe640 1 --2- 172.21.15.60:0/714605844 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc6f4037130 0x7fc6f4039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.473+0000 7fc7077fe640 1 --2- 172.21.15.60:0/714605844 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc6f4037130 0x7fc6f4039640 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.478+0000 7fc704ff9640 1 -- 172.21.15.60:0/714605844 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fc6f800d330 con 0x7fc70812ece0 2023-08-24T10:07:25.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.727+0000 7fc70e71e640 1 -- 172.21.15.60:0/714605844 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc708073730 con 0x7fc70812ece0 2023-08-24T10:07:25.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.728+0000 7fc704ff9640 1 -- 172.21.15.60:0/714605844 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7fc6f800d150 con 0x7fc70812ece0 2023-08-24T10:07:25.729 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:25.729 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":156,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:07:25.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.729+0000 7fc70e71e640 1 -- 172.21.15.60:0/714605844 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc6f4037130 msgr2=0x7fc6f4039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.729+0000 7fc70e71e640 1 --2- 172.21.15.60:0/714605844 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc6f4037130 0x7fc6f4039640 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.729+0000 7fc70e71e640 1 -- 172.21.15.60:0/714605844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc70812ece0 msgr2=0x7fc708070060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.729+0000 7fc70e71e640 1 --2- 172.21.15.60:0/714605844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc70812ece0 0x7fc708070060 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.729+0000 7fc70e71e640 1 -- 172.21.15.60:0/714605844 shutdown_connections 2023-08-24T10:07:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.729+0000 7fc70e71e640 1 --2- 172.21.15.60:0/714605844 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc6f4037130 0x7fc6f4039640 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.729+0000 7fc70e71e640 1 --2- 172.21.15.60:0/714605844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc70812ece0 0x7fc708070060 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:25.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.729+0000 7fc70e71e640 1 -- 172.21.15.60:0/714605844 >> 172.21.15.60:0/714605844 conn(0x7fc70809e110 msgr2=0x7fc708096820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:25.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.729+0000 7fc70e71e640 1 -- 172.21.15.60:0/714605844 shutdown_connections 2023-08-24T10:07:25.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.729+0000 7fc70e71e640 1 -- 172.21.15.60:0/714605844 wait complete. 2023-08-24T10:07:25.741 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:07:25.742 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:25.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.849+0000 7f8e10c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:25.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.853+0000 7f8e10c6c640 1 Processor -- start 2023-08-24T10:07:25.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.853+0000 7f8e10c6c640 1 -- start start 2023-08-24T10:07:25.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.854+0000 7f8e10c6c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8e0c12d040 0x7f8e0c1358b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:25.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.854+0000 7f8e10c6c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e0c0709e0 con 0x7f8e0c06fa30 2023-08-24T10:07:25.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.854+0000 7f8e10c6c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e0c12d8d0 con 0x7f8e0c12d040 2023-08-24T10:07:25.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.854+0000 7f8e09d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8e0c12d040 0x7f8e0c1358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:25.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.854+0000 7f8e09d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8e0c12d040 0x7f8e0c1358b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39330/0 (socket says 172.21.15.60:39330) 2023-08-24T10:07:25.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.854+0000 7f8e09d9b640 1 -- 172.21.15.60:0/3064438284 learned_addr learned my addr 172.21.15.60:0/3064438284 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:25.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.854+0000 7f8e0a59c640 1 --1- 172.21.15.60:0/3064438284 >> v1:172.21.15.60:6789/0 conn(0x7f8e0c06fa30 0x7f8e0c098cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56514/0 (socket says 172.21.15.60:56514) 2023-08-24T10:07:25.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.854+0000 7f8e09d9b640 1 -- 172.21.15.60:0/3064438284 >> v1:172.21.15.60:6789/0 conn(0x7f8e0c06fa30 legacy=0x7f8e0c098cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:25.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.854+0000 7f8e09d9b640 1 -- 172.21.15.60:0/3064438284 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0c12d5c0 con 0x7f8e0c12d040 2023-08-24T10:07:25.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.854+0000 7f8e09d9b640 1 --2- 172.21.15.60:0/3064438284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c12d040 0x7f8e0c1358b0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eabd6280c4364cb server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:25.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.855+0000 7f8e0959a640 1 -- 172.21.15.60:0/3064438284 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8df4009dd0 con 0x7f8e0c12d040 2023-08-24T10:07:25.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.855+0000 7f8e0959a640 1 -- 172.21.15.60:0/3064438284 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f8df400e230 con 0x7f8e0c12d040 2023-08-24T10:07:25.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.855+0000 7f8e0959a640 1 -- 172.21.15.60:0/3064438284 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8df400e4d0 con 0x7f8e0c12d040 2023-08-24T10:07:25.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.855+0000 7f8e10c6c640 1 -- 172.21.15.60:0/3064438284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c12d040 msgr2=0x7f8e0c1358b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:25.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.855+0000 7f8e10c6c640 1 --2- 172.21.15.60:0/3064438284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c12d040 0x7f8e0c1358b0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:25.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.855+0000 7f8e10c6c640 1 -- 172.21.15.60:0/3064438284 shutdown_connections 2023-08-24T10:07:25.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.855+0000 7f8e10c6c640 1 --2- 172.21.15.60:0/3064438284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c12d040 0x7f8e0c1358b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:25.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.855+0000 7f8e10c6c640 1 -- 172.21.15.60:0/3064438284 >> 172.21.15.60:0/3064438284 conn(0x7f8e0c0a2700 msgr2=0x7f8e0c0a2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:25.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.855+0000 7f8e10c6c640 1 -- 172.21.15.60:0/3064438284 shutdown_connections 2023-08-24T10:07:25.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.855+0000 7f8e10c6c640 1 -- 172.21.15.60:0/3064438284 wait complete. 2023-08-24T10:07:25.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.856+0000 7f8e10c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:25.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.857+0000 7f8e10c6c640 1 Processor -- start 2023-08-24T10:07:25.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.857+0000 7f8e10c6c640 1 -- start start 2023-08-24T10:07:25.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.857+0000 7f8e10c6c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c06fa30 0x7f8e0c12a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:25.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.857+0000 7f8e10c6c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c131050 con 0x7f8e0c06fa30 2023-08-24T10:07:25.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.857+0000 7f8e0a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c06fa30 0x7f8e0c12a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:25.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.858+0000 7f8e0a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c06fa30 0x7f8e0c12a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39342/0 (socket says 172.21.15.60:39342) 2023-08-24T10:07:25.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.858+0000 7f8e0a59c640 1 -- 172.21.15.60:0/1324903336 learned_addr learned my addr 172.21.15.60:0/1324903336 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:25.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.858+0000 7f8e0a59c640 1 -- 172.21.15.60:0/1324903336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0c06ca90 con 0x7f8e0c06fa30 2023-08-24T10:07:25.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.858+0000 7f8e0a59c640 1 --2- 172.21.15.60:0/1324903336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c06fa30 0x7f8e0c12a330 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:25.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.858+0000 7f8e02ffd640 1 -- 172.21.15.60:0/1324903336 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8e040071e0 con 0x7f8e0c06fa30 2023-08-24T10:07:25.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.858+0000 7f8e02ffd640 1 -- 172.21.15.60:0/1324903336 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f8e04007340 con 0x7f8e0c06fa30 2023-08-24T10:07:25.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.858+0000 7f8e10c6c640 1 -- 172.21.15.60:0/1324903336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e0c12fe30 con 0x7f8e0c06fa30 2023-08-24T10:07:25.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.858+0000 7f8e02ffd640 1 -- 172.21.15.60:0/1324903336 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8e040075e0 con 0x7f8e0c06fa30 2023-08-24T10:07:25.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.858+0000 7f8e10c6c640 1 -- 172.21.15.60:0/1324903336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0c12fab0 con 0x7f8e0c06fa30 2023-08-24T10:07:25.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.859+0000 7f8e02ffd640 1 -- 172.21.15.60:0/1324903336 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f8e040057e0 con 0x7f8e0c06fa30 2023-08-24T10:07:25.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.859+0000 7f8e02ffd640 1 --2- 172.21.15.60:0/1324903336 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8dec037130 0x7f8dec039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:25.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.859+0000 7f8e02ffd640 1 -- 172.21.15.60:0/1324903336 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f8e040463b0 con 0x7f8e0c06fa30 2023-08-24T10:07:25.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.859+0000 7f8e10c6c640 1 -- 172.21.15.60:0/1324903336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e0c06c820 con 0x7f8e0c06fa30 2023-08-24T10:07:25.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.859+0000 7f8e09d9b640 1 --2- 172.21.15.60:0/1324903336 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8dec037130 0x7f8dec039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:25.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.860+0000 7f8e09d9b640 1 --2- 172.21.15.60:0/1324903336 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8dec037130 0x7f8dec039640 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:25.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:25.864+0000 7f8e02ffd640 1 -- 172.21.15.60:0/1324903336 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f8e0400d330 con 0x7f8e0c06fa30 2023-08-24T10:07:26.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:26.037+0000 7f8e10c6c640 1 -- 172.21.15.60:0/1324903336 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e0c061800 con 0x7f8dec037130 2023-08-24T10:07:26.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:26.038+0000 7f8e02ffd640 1 -- 172.21.15.60:0/1324903336 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f8e0c061800 con 0x7f8dec037130 2023-08-24T10:07:26.039 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:26.040 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:26.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:26.040+0000 7f8e10c6c640 1 -- 172.21.15.60:0/1324903336 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8dec037130 msgr2=0x7f8dec039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:26.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:26.040+0000 7f8e10c6c640 1 --2- 172.21.15.60:0/1324903336 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8dec037130 0x7f8dec039640 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:26.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:26.040+0000 7f8e10c6c640 1 -- 172.21.15.60:0/1324903336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c06fa30 msgr2=0x7f8e0c12a330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:26.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:26.040+0000 7f8e10c6c640 1 --2- 172.21.15.60:0/1324903336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c06fa30 0x7f8e0c12a330 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:26.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:26.040+0000 7f8e10c6c640 1 -- 172.21.15.60:0/1324903336 shutdown_connections 2023-08-24T10:07:26.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:26.040+0000 7f8e10c6c640 1 --2- 172.21.15.60:0/1324903336 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8dec037130 0x7f8dec039640 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:26.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:26.040+0000 7f8e10c6c640 1 --2- 172.21.15.60:0/1324903336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c06fa30 0x7f8e0c12a330 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:26.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:26.040+0000 7f8e10c6c640 1 -- 172.21.15.60:0/1324903336 >> 172.21.15.60:0/1324903336 conn(0x7f8e0c0a2700 msgr2=0x7f8e0c1b87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:26.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:26.040+0000 7f8e10c6c640 1 -- 172.21.15.60:0/1324903336 shutdown_connections 2023-08-24T10:07:26.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:26.040+0000 7f8e10c6c640 1 -- 172.21.15.60:0/1324903336 wait complete. 2023-08-24T10:07:26.052 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2023-08-24T10:07:25.524246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:27.950 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~140s 2023-08-24T10:07:27.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~140s 2023-08-24T10:07:27.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~140s 2023-08-24T10:07:27.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~140s 2023-08-24T10:07:29.054 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:29.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.174+0000 7f4fca39e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.178+0000 7f4fca39e640 1 Processor -- start 2023-08-24T10:07:29.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.178+0000 7f4fca39e640 1 -- start start 2023-08-24T10:07:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.179+0000 7f4fca39e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4fc4131d40 0x7f4fc4134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.179+0000 7f4fca39e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc4099440 con 0x7f4fc406d300 2023-08-24T10:07:29.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.179+0000 7f4fca39e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc409a050 con 0x7f4fc4131d40 2023-08-24T10:07:29.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.179+0000 7f4fc37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4fc4131d40 0x7f4fc4134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:29.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.179+0000 7f4fc37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4fc4131d40 0x7f4fc4134160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40340/0 (socket says 172.21.15.60:40340) 2023-08-24T10:07:29.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.179+0000 7f4fc37fe640 1 -- 172.21.15.60:0/2751264410 learned_addr learned my addr 172.21.15.60:0/2751264410 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:29.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.179+0000 7f4fc37fe640 1 -- 172.21.15.60:0/2751264410 >> v1:172.21.15.60:6789/0 conn(0x7f4fc406d300 legacy=0x7f4fc412bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:29.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.179+0000 7f4fc37fe640 1 -- 172.21.15.60:0/2751264410 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc412b9d0 con 0x7f4fc4131d40 2023-08-24T10:07:29.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.180+0000 7f4fc37fe640 1 --2- 172.21.15.60:0/2751264410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc4131d40 0x7f4fc4134160 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c8c564eb0c7596 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:29.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.180+0000 7f4fc2ffd640 1 -- 172.21.15.60:0/2751264410 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4fb400ad50 con 0x7f4fc4131d40 2023-08-24T10:07:29.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.180+0000 7f4fc2ffd640 1 -- 172.21.15.60:0/2751264410 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4fb400f340 con 0x7f4fc4131d40 2023-08-24T10:07:29.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.180+0000 7f4fc2ffd640 1 -- 172.21.15.60:0/2751264410 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4fb400f5e0 con 0x7f4fc4131d40 2023-08-24T10:07:29.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.180+0000 7f4fca39e640 1 -- 172.21.15.60:0/2751264410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc4131d40 msgr2=0x7f4fc4134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:29.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.180+0000 7f4fca39e640 1 --2- 172.21.15.60:0/2751264410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc4131d40 0x7f4fc4134160 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:29.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.180+0000 7f4fca39e640 1 -- 172.21.15.60:0/2751264410 shutdown_connections 2023-08-24T10:07:29.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.180+0000 7f4fca39e640 1 --2- 172.21.15.60:0/2751264410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc4131d40 0x7f4fc4134160 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:29.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.181+0000 7f4fca39e640 1 -- 172.21.15.60:0/2751264410 >> 172.21.15.60:0/2751264410 conn(0x7f4fc409e110 msgr2=0x7f4fc409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:29.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.181+0000 7f4fca39e640 1 -- 172.21.15.60:0/2751264410 shutdown_connections 2023-08-24T10:07:29.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.181+0000 7f4fca39e640 1 -- 172.21.15.60:0/2751264410 wait complete. 2023-08-24T10:07:29.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.181+0000 7f4fca39e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:29.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.182+0000 7f4fca39e640 1 Processor -- start 2023-08-24T10:07:29.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.182+0000 7f4fca39e640 1 -- start start 2023-08-24T10:07:29.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.183+0000 7f4fca39e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc406d300 0x7f4fc40968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:29.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.183+0000 7f4fca39e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc4071410 con 0x7f4fc406d300 2023-08-24T10:07:29.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.183+0000 7f4fc3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc406d300 0x7f4fc40968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:29.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.183+0000 7f4fc3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc406d300 0x7f4fc40968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40346/0 (socket says 172.21.15.60:40346) 2023-08-24T10:07:29.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.183+0000 7f4fc3fff640 1 -- 172.21.15.60:0/3115069675 learned_addr learned my addr 172.21.15.60:0/3115069675 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:29.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.183+0000 7f4fc3fff640 1 -- 172.21.15.60:0/3115069675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc406f480 con 0x7f4fc406d300 2023-08-24T10:07:29.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.183+0000 7f4fc3fff640 1 --2- 172.21.15.60:0/3115069675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc406d300 0x7f4fc40968d0 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:29.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.183+0000 7f4fc0ff9640 1 -- 172.21.15.60:0/3115069675 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4fac00c860 con 0x7f4fc406d300 2023-08-24T10:07:29.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.183+0000 7f4fc0ff9640 1 -- 172.21.15.60:0/3115069675 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4fac00c9c0 con 0x7f4fc406d300 2023-08-24T10:07:29.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.183+0000 7f4fca39e640 1 -- 172.21.15.60:0/3115069675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc4071810 con 0x7f4fc406d300 2023-08-24T10:07:29.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.183+0000 7f4fc0ff9640 1 -- 172.21.15.60:0/3115069675 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4fac00cc60 con 0x7f4fc406d300 2023-08-24T10:07:29.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.183+0000 7f4fca39e640 1 -- 172.21.15.60:0/3115069675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc406e2d0 con 0x7f4fc406d300 2023-08-24T10:07:29.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.184+0000 7f4fc0ff9640 1 -- 172.21.15.60:0/3115069675 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f4fac011310 con 0x7f4fc406d300 2023-08-24T10:07:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.185+0000 7f4fca39e640 1 -- 172.21.15.60:0/3115069675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc406d9f0 con 0x7f4fc406d300 2023-08-24T10:07:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.185+0000 7f4fc0ff9640 1 --2- 172.21.15.60:0/3115069675 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4fa4037180 0x7f4fa4039690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.185+0000 7f4fc0ff9640 1 -- 172.21.15.60:0/3115069675 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f4fac046a20 con 0x7f4fc406d300 2023-08-24T10:07:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.185+0000 7f4fc37fe640 1 --2- 172.21.15.60:0/3115069675 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4fa4037180 0x7f4fa4039690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.185+0000 7f4fc37fe640 1 --2- 172.21.15.60:0/3115069675 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4fa4037180 0x7f4fa4039690 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:29.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.189+0000 7f4fc0ff9640 1 -- 172.21.15.60:0/3115069675 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f4fac011e60 con 0x7f4fc406d300 2023-08-24T10:07:29.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.361+0000 7f4fca39e640 1 -- 172.21.15.60:0/3115069675 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4fc412a310 con 0x7f4fa4037180 2023-08-24T10:07:29.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.363+0000 7f4fc0ff9640 1 -- 172.21.15.60:0/3115069675 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f4fb400ad20 con 0x7f4fa4037180 2023-08-24T10:07:29.364 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:29.364 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:29.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.364+0000 7f4fca39e640 1 -- 172.21.15.60:0/3115069675 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4fa4037180 msgr2=0x7f4fa4039690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:29.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.364+0000 7f4fca39e640 1 --2- 172.21.15.60:0/3115069675 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4fa4037180 0x7f4fa4039690 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:29.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.364+0000 7f4fca39e640 1 -- 172.21.15.60:0/3115069675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc406d300 msgr2=0x7f4fc40968d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:29.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.364+0000 7f4fca39e640 1 --2- 172.21.15.60:0/3115069675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc406d300 0x7f4fc40968d0 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:29.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.365+0000 7f4fca39e640 1 -- 172.21.15.60:0/3115069675 shutdown_connections 2023-08-24T10:07:29.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.365+0000 7f4fca39e640 1 --2- 172.21.15.60:0/3115069675 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4fa4037180 0x7f4fa4039690 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:29.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.365+0000 7f4fca39e640 1 --2- 172.21.15.60:0/3115069675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc406d300 0x7f4fc40968d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:29.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.365+0000 7f4fca39e640 1 -- 172.21.15.60:0/3115069675 >> 172.21.15.60:0/3115069675 conn(0x7f4fc409e110 msgr2=0x7f4fc412ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:29.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.365+0000 7f4fca39e640 1 -- 172.21.15.60:0/3115069675 shutdown_connections 2023-08-24T10:07:29.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.365+0000 7f4fca39e640 1 -- 172.21.15.60:0/3115069675 wait complete. 2023-08-24T10:07:29.378 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2023-08-24T10:07:27.524635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:29.379 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:29.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.486+0000 7ff00439e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:29.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.490+0000 7ff00439e640 1 Processor -- start 2023-08-24T10:07:29.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.490+0000 7ff00439e640 1 -- start start 2023-08-24T10:07:29.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.491+0000 7ff00439e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feffc070110 0x7feffc1356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:29.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.491+0000 7ff00439e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feffc096f10 con 0x7feffc071e40 2023-08-24T10:07:29.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.491+0000 7ff00439e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feffc0977f0 con 0x7feffc070110 2023-08-24T10:07:29.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.491+0000 7ff001939640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feffc070110 0x7feffc1356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:29.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.491+0000 7ff001939640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feffc070110 0x7feffc1356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40354/0 (socket says 172.21.15.60:40354) 2023-08-24T10:07:29.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.491+0000 7ff00213a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7feffc071e40 0x7feffc096800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38048/0 (socket says 172.21.15.60:38048) 2023-08-24T10:07:29.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.491+0000 7ff001939640 1 -- 172.21.15.60:0/2815559090 learned_addr learned my addr 172.21.15.60:0/2815559090 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:29.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.491+0000 7ff001939640 1 -- 172.21.15.60:0/2815559090 >> v1:172.21.15.60:6789/0 conn(0x7feffc071e40 legacy=0x7feffc096800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:29.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.491+0000 7ff001939640 1 -- 172.21.15.60:0/2815559090 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc09a280 con 0x7feffc070110 2023-08-24T10:07:29.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.492+0000 7ff001939640 1 --2- 172.21.15.60:0/2815559090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc070110 0x7feffc1356a0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f6036190d9a1fa server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:29.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.492+0000 7ff001138640 1 -- 172.21.15.60:0/2815559090 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7feff0009dd0 con 0x7feffc070110 2023-08-24T10:07:29.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.492+0000 7ff001138640 1 -- 172.21.15.60:0/2815559090 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7feff000e230 con 0x7feffc070110 2023-08-24T10:07:29.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.492+0000 7ff001138640 1 -- 172.21.15.60:0/2815559090 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7feff000e4d0 con 0x7feffc070110 2023-08-24T10:07:29.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.492+0000 7ff00439e640 1 -- 172.21.15.60:0/2815559090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc070110 msgr2=0x7feffc1356a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:29.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.492+0000 7ff00439e640 1 --2- 172.21.15.60:0/2815559090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc070110 0x7feffc1356a0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:29.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.492+0000 7ff00439e640 1 -- 172.21.15.60:0/2815559090 shutdown_connections 2023-08-24T10:07:29.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.492+0000 7ff00439e640 1 --2- 172.21.15.60:0/2815559090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc070110 0x7feffc1356a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:29.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.492+0000 7ff00439e640 1 -- 172.21.15.60:0/2815559090 >> 172.21.15.60:0/2815559090 conn(0x7feffc09e110 msgr2=0x7feffc09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:29.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.492+0000 7ff00439e640 1 -- 172.21.15.60:0/2815559090 shutdown_connections 2023-08-24T10:07:29.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.492+0000 7ff00439e640 1 -- 172.21.15.60:0/2815559090 wait complete. 2023-08-24T10:07:29.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.493+0000 7ff00439e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:29.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.493+0000 7ff00439e640 1 Processor -- start 2023-08-24T10:07:29.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.494+0000 7ff00439e640 1 -- start start 2023-08-24T10:07:29.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.494+0000 7ff00439e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc070110 0x7feffc093460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:29.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.494+0000 7ff00439e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feffc08ff10 con 0x7feffc070110 2023-08-24T10:07:29.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.494+0000 7ff00213a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc070110 0x7feffc093460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:29.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.494+0000 7ff00213a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc070110 0x7feffc093460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40364/0 (socket says 172.21.15.60:40364) 2023-08-24T10:07:29.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.494+0000 7ff00213a640 1 -- 172.21.15.60:0/52666454 learned_addr learned my addr 172.21.15.60:0/52666454 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:29.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.495+0000 7ff00213a640 1 -- 172.21.15.60:0/52666454 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc090680 con 0x7feffc070110 2023-08-24T10:07:29.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.495+0000 7ff00213a640 1 --2- 172.21.15.60:0/52666454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc070110 0x7feffc093460 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:29.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.495+0000 7feffaffd640 1 -- 172.21.15.60:0/52666454 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7feff40071e0 con 0x7feffc070110 2023-08-24T10:07:29.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.495+0000 7feffaffd640 1 -- 172.21.15.60:0/52666454 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7feff4007340 con 0x7feffc070110 2023-08-24T10:07:29.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.495+0000 7ff00439e640 1 -- 172.21.15.60:0/52666454 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feffc092e80 con 0x7feffc070110 2023-08-24T10:07:29.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.495+0000 7feffaffd640 1 -- 172.21.15.60:0/52666454 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7feff40075e0 con 0x7feffc070110 2023-08-24T10:07:29.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.495+0000 7ff00439e640 1 -- 172.21.15.60:0/52666454 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feffc092b70 con 0x7feffc070110 2023-08-24T10:07:29.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.496+0000 7feffaffd640 1 -- 172.21.15.60:0/52666454 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7feff40057e0 con 0x7feffc070110 2023-08-24T10:07:29.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.496+0000 7ff00439e640 1 -- 172.21.15.60:0/52666454 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feffc061800 con 0x7feffc070110 2023-08-24T10:07:29.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.496+0000 7feffaffd640 1 --2- 172.21.15.60:0/52666454 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fefe8037130 0x7fefe8039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:29.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.496+0000 7feffaffd640 1 -- 172.21.15.60:0/52666454 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7feff40463f0 con 0x7feffc070110 2023-08-24T10:07:29.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.497+0000 7ff001939640 1 --2- 172.21.15.60:0/52666454 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fefe8037130 0x7fefe8039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:29.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.497+0000 7ff001939640 1 --2- 172.21.15.60:0/52666454 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fefe8037130 0x7fefe8039640 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:29.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.501+0000 7feffaffd640 1 -- 172.21.15.60:0/52666454 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7feff400d330 con 0x7feffc070110 2023-08-24T10:07:29.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.672+0000 7ff00439e640 1 -- 172.21.15.60:0/52666454 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feffc0932f0 con 0x7fefe8037130 2023-08-24T10:07:29.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.674+0000 7feffaffd640 1 -- 172.21.15.60:0/52666454 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7feffc0932f0 con 0x7fefe8037130 2023-08-24T10:07:29.675 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:29.675 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:29.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.675+0000 7ff00439e640 1 -- 172.21.15.60:0/52666454 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fefe8037130 msgr2=0x7fefe8039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:29.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.675+0000 7ff00439e640 1 --2- 172.21.15.60:0/52666454 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fefe8037130 0x7fefe8039640 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:29.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.675+0000 7ff00439e640 1 -- 172.21.15.60:0/52666454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc070110 msgr2=0x7feffc093460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:29.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.675+0000 7ff00439e640 1 --2- 172.21.15.60:0/52666454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc070110 0x7feffc093460 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:29.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.675+0000 7ff00439e640 1 -- 172.21.15.60:0/52666454 shutdown_connections 2023-08-24T10:07:29.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.675+0000 7ff00439e640 1 --2- 172.21.15.60:0/52666454 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fefe8037130 0x7fefe8039640 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:29.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.675+0000 7ff00439e640 1 --2- 172.21.15.60:0/52666454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feffc070110 0x7feffc093460 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:29.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.675+0000 7ff00439e640 1 -- 172.21.15.60:0/52666454 >> 172.21.15.60:0/52666454 conn(0x7feffc09e110 msgr2=0x7feffc098490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:29.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.675+0000 7ff00439e640 1 -- 172.21.15.60:0/52666454 shutdown_connections 2023-08-24T10:07:29.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.675+0000 7ff00439e640 1 -- 172.21.15.60:0/52666454 wait complete. 2023-08-24T10:07:29.687 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2023-08-24T10:07:29.524974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:29.688 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:07:29.688 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:29.688 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:07:29.688 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:29.689 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:07:29.689 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:29.689 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:07:29.689 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:29.690 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:07:29.690 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:29.690 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:07:29.690 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:29.691 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:07:29.691 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:29.691 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:07:29.691 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:29.692 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:07:29.692 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:29.692 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:07:29.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.799+0000 7faf1bd1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:29.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.802+0000 7faf1bd1e640 1 Processor -- start 2023-08-24T10:07:29.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.802+0000 7faf1bd1e640 1 -- start start 2023-08-24T10:07:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.802+0000 7faf1bd1e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faf1412ece0 0x7faf1412f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.802+0000 7faf1bd1e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf1409a650 con 0x7faf1412f630 2023-08-24T10:07:29.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.802+0000 7faf1bd1e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf1406da00 con 0x7faf1412ece0 2023-08-24T10:07:29.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.803+0000 7faf19aba640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faf1412ece0 0x7faf1412f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:29.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.803+0000 7faf19aba640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faf1412ece0 0x7faf1412f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40380/0 (socket says 172.21.15.60:40380) 2023-08-24T10:07:29.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.803+0000 7faf19aba640 1 -- 172.21.15.60:0/1962785590 learned_addr learned my addr 172.21.15.60:0/1962785590 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:29.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.803+0000 7faf19aba640 1 -- 172.21.15.60:0/1962785590 >> v1:172.21.15.60:6789/0 conn(0x7faf1412f630 legacy=0x7faf14097fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:29.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.803+0000 7faf19aba640 1 -- 172.21.15.60:0/1962785590 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1412ea10 con 0x7faf1412ece0 2023-08-24T10:07:29.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.803+0000 7faf19aba640 1 --2- 172.21.15.60:0/1962785590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf1412ece0 0x7faf1412f0f0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ee619930dd31e1 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:29.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.803+0000 7faf18ab8640 1 -- 172.21.15.60:0/1962785590 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7faf1000ad60 con 0x7faf1412ece0 2023-08-24T10:07:29.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.803+0000 7faf18ab8640 1 -- 172.21.15.60:0/1962785590 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7faf1000f340 con 0x7faf1412ece0 2023-08-24T10:07:29.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.803+0000 7faf18ab8640 1 -- 172.21.15.60:0/1962785590 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7faf1000f5e0 con 0x7faf1412ece0 2023-08-24T10:07:29.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.804+0000 7faf1bd1e640 1 -- 172.21.15.60:0/1962785590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf1412ece0 msgr2=0x7faf1412f0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:29.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.804+0000 7faf1bd1e640 1 --2- 172.21.15.60:0/1962785590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf1412ece0 0x7faf1412f0f0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:29.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.804+0000 7faf1bd1e640 1 -- 172.21.15.60:0/1962785590 shutdown_connections 2023-08-24T10:07:29.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.804+0000 7faf1bd1e640 1 --2- 172.21.15.60:0/1962785590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf1412ece0 0x7faf1412f0f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:29.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.804+0000 7faf1bd1e640 1 -- 172.21.15.60:0/1962785590 >> 172.21.15.60:0/1962785590 conn(0x7faf1409e110 msgr2=0x7faf1409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:29.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.804+0000 7faf1bd1e640 1 -- 172.21.15.60:0/1962785590 shutdown_connections 2023-08-24T10:07:29.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.804+0000 7faf1bd1e640 1 -- 172.21.15.60:0/1962785590 wait complete. 2023-08-24T10:07:29.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.804+0000 7faf1bd1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:29.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.805+0000 7faf1bd1e640 1 Processor -- start 2023-08-24T10:07:29.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.805+0000 7faf1bd1e640 1 -- start start 2023-08-24T10:07:29.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.806+0000 7faf1bd1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf1412ece0 0x7faf14129280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:29.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.806+0000 7faf1bd1e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf141288f0 con 0x7faf1412ece0 2023-08-24T10:07:29.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.806+0000 7faf19aba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf1412ece0 0x7faf14129280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:29.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.806+0000 7faf19aba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf1412ece0 0x7faf14129280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40392/0 (socket says 172.21.15.60:40392) 2023-08-24T10:07:29.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.806+0000 7faf19aba640 1 -- 172.21.15.60:0/3480013886 learned_addr learned my addr 172.21.15.60:0/3480013886 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:29.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.806+0000 7faf19aba640 1 -- 172.21.15.60:0/3480013886 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf14128ec0 con 0x7faf1412ece0 2023-08-24T10:07:29.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.806+0000 7faf19aba640 1 --2- 172.21.15.60:0/3480013886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf1412ece0 0x7faf14129280 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:29.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.806+0000 7faf0a7fc640 1 -- 172.21.15.60:0/3480013886 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7faf1000fd70 con 0x7faf1412ece0 2023-08-24T10:07:29.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.806+0000 7faf0a7fc640 1 -- 172.21.15.60:0/3480013886 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7faf1000fed0 con 0x7faf1412ece0 2023-08-24T10:07:29.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.806+0000 7faf1bd1e640 1 -- 172.21.15.60:0/3480013886 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf1412aed0 con 0x7faf1412ece0 2023-08-24T10:07:29.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.806+0000 7faf0a7fc640 1 -- 172.21.15.60:0/3480013886 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7faf10011550 con 0x7faf1412ece0 2023-08-24T10:07:29.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.806+0000 7faf1bd1e640 1 -- 172.21.15.60:0/3480013886 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf1408da20 con 0x7faf1412ece0 2023-08-24T10:07:29.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.807+0000 7faf0a7fc640 1 -- 172.21.15.60:0/3480013886 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7faf10011700 con 0x7faf1412ece0 2023-08-24T10:07:29.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.807+0000 7faf1bd1e640 1 -- 172.21.15.60:0/3480013886 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf14061800 con 0x7faf1412ece0 2023-08-24T10:07:29.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.808+0000 7faf0a7fc640 1 --2- 172.21.15.60:0/3480013886 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7faef8037130 0x7faef8039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:29.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.808+0000 7faf0a7fc640 1 -- 172.21.15.60:0/3480013886 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7faf10017040 con 0x7faf1412ece0 2023-08-24T10:07:29.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.808+0000 7faf192b9640 1 --2- 172.21.15.60:0/3480013886 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7faef8037130 0x7faef8039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:29.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.808+0000 7faf192b9640 1 --2- 172.21.15.60:0/3480013886 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7faef8037130 0x7faef8039640 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:29.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:29.812+0000 7faf0a7fc640 1 -- 172.21.15.60:0/3480013886 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7faf100103b0 con 0x7faf1412ece0 2023-08-24T10:07:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.057+0000 7faf1bd1e640 1 -- 172.21.15.60:0/3480013886 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7faf14073730 con 0x7faf1412ece0 2023-08-24T10:07:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.057+0000 7faf0a7fc640 1 -- 172.21.15.60:0/3480013886 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7faf1000b3c0 con 0x7faf1412ece0 2023-08-24T10:07:30.059 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:30.059 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":160,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:07:30.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.059+0000 7faf1bd1e640 1 -- 172.21.15.60:0/3480013886 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7faef8037130 msgr2=0x7faef8039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:30.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.059+0000 7faf1bd1e640 1 --2- 172.21.15.60:0/3480013886 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7faef8037130 0x7faef8039640 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:30.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.059+0000 7faf1bd1e640 1 -- 172.21.15.60:0/3480013886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf1412ece0 msgr2=0x7faf14129280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:30.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.059+0000 7faf1bd1e640 1 --2- 172.21.15.60:0/3480013886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf1412ece0 0x7faf14129280 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:30.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.059+0000 7faf1bd1e640 1 -- 172.21.15.60:0/3480013886 shutdown_connections 2023-08-24T10:07:30.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.059+0000 7faf1bd1e640 1 --2- 172.21.15.60:0/3480013886 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7faef8037130 0x7faef8039640 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:30.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.059+0000 7faf1bd1e640 1 --2- 172.21.15.60:0/3480013886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf1412ece0 0x7faf14129280 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:30.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.059+0000 7faf1bd1e640 1 -- 172.21.15.60:0/3480013886 >> 172.21.15.60:0/3480013886 conn(0x7faf1409e110 msgr2=0x7faf14097710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:30.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.059+0000 7faf1bd1e640 1 -- 172.21.15.60:0/3480013886 shutdown_connections 2023-08-24T10:07:30.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.059+0000 7faf1bd1e640 1 -- 172.21.15.60:0/3480013886 wait complete. 2023-08-24T10:07:30.071 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:07:30.071 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.178+0000 7ff3c286c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.182+0000 7ff3c286c640 1 Processor -- start 2023-08-24T10:07:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.182+0000 7ff3c286c640 1 -- start start 2023-08-24T10:07:30.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.182+0000 7ff3c286c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff3bc132630 0x7ff3bc134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:30.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.182+0000 7ff3c286c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3bc06e320 con 0x7ff3bc096860 2023-08-24T10:07:30.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.182+0000 7ff3c286c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3bc06db70 con 0x7ff3bc132630 2023-08-24T10:07:30.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.183+0000 7ff3bb7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff3bc132630 0x7ff3bc134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:30.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.183+0000 7ff3bb7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff3bc132630 0x7ff3bc134a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40404/0 (socket says 172.21.15.60:40404) 2023-08-24T10:07:30.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.183+0000 7ff3bb7fe640 1 -- 172.21.15.60:0/4188536972 learned_addr learned my addr 172.21.15.60:0/4188536972 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:30.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.183+0000 7ff3bb7fe640 1 -- 172.21.15.60:0/4188536972 >> v1:172.21.15.60:6789/0 conn(0x7ff3bc096860 legacy=0x7ff3bc096c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:30.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.183+0000 7ff3bb7fe640 1 -- 172.21.15.60:0/4188536972 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc06cef0 con 0x7ff3bc132630 2023-08-24T10:07:30.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.183+0000 7ff3bb7fe640 1 --2- 172.21.15.60:0/4188536972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3bc132630 0x7ff3bc134a50 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e9377a24cf23f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:30.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.183+0000 7ff3baffd640 1 -- 172.21.15.60:0/4188536972 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff3b0009dd0 con 0x7ff3bc132630 2023-08-24T10:07:30.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.183+0000 7ff3baffd640 1 -- 172.21.15.60:0/4188536972 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ff3b000e230 con 0x7ff3bc132630 2023-08-24T10:07:30.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.183+0000 7ff3baffd640 1 -- 172.21.15.60:0/4188536972 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff3b000e4d0 con 0x7ff3bc132630 2023-08-24T10:07:30.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.184+0000 7ff3c286c640 1 -- 172.21.15.60:0/4188536972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3bc132630 msgr2=0x7ff3bc134a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:30.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.184+0000 7ff3c286c640 1 --2- 172.21.15.60:0/4188536972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3bc132630 0x7ff3bc134a50 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:30.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.184+0000 7ff3c286c640 1 -- 172.21.15.60:0/4188536972 shutdown_connections 2023-08-24T10:07:30.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.184+0000 7ff3c286c640 1 --2- 172.21.15.60:0/4188536972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3bc132630 0x7ff3bc134a50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:30.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.184+0000 7ff3c286c640 1 -- 172.21.15.60:0/4188536972 >> 172.21.15.60:0/4188536972 conn(0x7ff3bc09e110 msgr2=0x7ff3bc09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:30.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.184+0000 7ff3c286c640 1 -- 172.21.15.60:0/4188536972 shutdown_connections 2023-08-24T10:07:30.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.184+0000 7ff3c286c640 1 -- 172.21.15.60:0/4188536972 wait complete. 2023-08-24T10:07:30.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.184+0000 7ff3c286c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:30.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.185+0000 7ff3c286c640 1 Processor -- start 2023-08-24T10:07:30.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.185+0000 7ff3c286c640 1 -- start start 2023-08-24T10:07:30.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.186+0000 7ff3c286c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3bc096860 0x7ff3bc127b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:30.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.186+0000 7ff3c286c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc129530 con 0x7ff3bc096860 2023-08-24T10:07:30.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.186+0000 7ff3bbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3bc096860 0x7ff3bc127b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:30.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.186+0000 7ff3bbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3bc096860 0x7ff3bc127b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40418/0 (socket says 172.21.15.60:40418) 2023-08-24T10:07:30.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.186+0000 7ff3bbfff640 1 -- 172.21.15.60:0/2119679965 learned_addr learned my addr 172.21.15.60:0/2119679965 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:30.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.186+0000 7ff3bbfff640 1 -- 172.21.15.60:0/2119679965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc1290c0 con 0x7ff3bc096860 2023-08-24T10:07:30.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.186+0000 7ff3bbfff640 1 --2- 172.21.15.60:0/2119679965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3bc096860 0x7ff3bc127b60 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:30.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.186+0000 7ff3b8ff9640 1 -- 172.21.15.60:0/2119679965 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff3ac00c7e0 con 0x7ff3bc096860 2023-08-24T10:07:30.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.187+0000 7ff3b8ff9640 1 -- 172.21.15.60:0/2119679965 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ff3ac00c940 con 0x7ff3bc096860 2023-08-24T10:07:30.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.187+0000 7ff3c286c640 1 -- 172.21.15.60:0/2119679965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3bc12efa0 con 0x7ff3bc096860 2023-08-24T10:07:30.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.187+0000 7ff3b8ff9640 1 -- 172.21.15.60:0/2119679965 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff3ac00cbe0 con 0x7ff3bc096860 2023-08-24T10:07:30.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.187+0000 7ff3c286c640 1 -- 172.21.15.60:0/2119679965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3bc097fa0 con 0x7ff3bc096860 2023-08-24T10:07:30.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.187+0000 7ff3b8ff9640 1 -- 172.21.15.60:0/2119679965 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7ff3ac011310 con 0x7ff3bc096860 2023-08-24T10:07:30.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.187+0000 7ff3c286c640 1 -- 172.21.15.60:0/2119679965 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3bc061800 con 0x7ff3bc096860 2023-08-24T10:07:30.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.188+0000 7ff3b8ff9640 1 --2- 172.21.15.60:0/2119679965 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff3a80370e0 0x7ff3a80395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:30.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.188+0000 7ff3b8ff9640 1 -- 172.21.15.60:0/2119679965 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7ff3ac046a50 con 0x7ff3bc096860 2023-08-24T10:07:30.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.188+0000 7ff3bb7fe640 1 --2- 172.21.15.60:0/2119679965 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff3a80370e0 0x7ff3a80395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:30.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.189+0000 7ff3bb7fe640 1 --2- 172.21.15.60:0/2119679965 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff3a80370e0 0x7ff3a80395f0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:30.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.192+0000 7ff3b8ff9640 1 -- 172.21.15.60:0/2119679965 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7ff3ac048e60 con 0x7ff3bc096860 2023-08-24T10:07:30.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.363+0000 7ff3c286c640 1 -- 172.21.15.60:0/2119679965 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff3bc06d190 con 0x7ff3a80370e0 2023-08-24T10:07:30.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.365+0000 7ff3b8ff9640 1 -- 172.21.15.60:0/2119679965 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7ff3b0009df0 con 0x7ff3a80370e0 2023-08-24T10:07:30.366 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:30.367 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:30.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.366+0000 7ff3c286c640 1 -- 172.21.15.60:0/2119679965 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff3a80370e0 msgr2=0x7ff3a80395f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:30.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.366+0000 7ff3c286c640 1 --2- 172.21.15.60:0/2119679965 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff3a80370e0 0x7ff3a80395f0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:30.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.367+0000 7ff3c286c640 1 -- 172.21.15.60:0/2119679965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3bc096860 msgr2=0x7ff3bc127b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:30.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.367+0000 7ff3c286c640 1 --2- 172.21.15.60:0/2119679965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3bc096860 0x7ff3bc127b60 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:30.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.367+0000 7ff3c286c640 1 -- 172.21.15.60:0/2119679965 shutdown_connections 2023-08-24T10:07:30.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.367+0000 7ff3c286c640 1 --2- 172.21.15.60:0/2119679965 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff3a80370e0 0x7ff3a80395f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:30.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.367+0000 7ff3c286c640 1 --2- 172.21.15.60:0/2119679965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3bc096860 0x7ff3bc127b60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:30.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.367+0000 7ff3c286c640 1 -- 172.21.15.60:0/2119679965 >> 172.21.15.60:0/2119679965 conn(0x7ff3bc09e110 msgr2=0x7ff3bc0950c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:30.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.367+0000 7ff3c286c640 1 -- 172.21.15.60:0/2119679965 shutdown_connections 2023-08-24T10:07:30.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:30.367+0000 7ff3c286c640 1 -- 172.21.15.60:0/2119679965 wait complete. 2023-08-24T10:07:30.379 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2023-08-24T10:07:29.524974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:33.153 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~146s 2023-08-24T10:07:33.154 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~146s 2023-08-24T10:07:33.154 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~146s 2023-08-24T10:07:33.154 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~146s 2023-08-24T10:07:33.381 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.503+0000 7fa0c839e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:33.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.507+0000 7fa0c839e640 1 Processor -- start 2023-08-24T10:07:33.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.507+0000 7fa0c839e640 1 -- start start 2023-08-24T10:07:33.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.508+0000 7fa0c839e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa0c0132630 0x7fa0c0134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:33.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.508+0000 7fa0c839e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c006e320 con 0x7fa0c0096860 2023-08-24T10:07:33.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.508+0000 7fa0c839e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c006db70 con 0x7fa0c0132630 2023-08-24T10:07:33.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.508+0000 7fa0c5939640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa0c0132630 0x7fa0c0134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:33.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.508+0000 7fa0c5939640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa0c0132630 0x7fa0c0134a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40424/0 (socket says 172.21.15.60:40424) 2023-08-24T10:07:33.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.508+0000 7fa0c5939640 1 -- 172.21.15.60:0/623059670 learned_addr learned my addr 172.21.15.60:0/623059670 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:33.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.508+0000 7fa0c5939640 1 -- 172.21.15.60:0/623059670 >> v1:172.21.15.60:6789/0 conn(0x7fa0c0096860 legacy=0x7fa0c0096c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:33.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.508+0000 7fa0c5939640 1 -- 172.21.15.60:0/623059670 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c006cef0 con 0x7fa0c0132630 2023-08-24T10:07:33.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.508+0000 7fa0c5939640 1 --2- 172.21.15.60:0/623059670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c0132630 0x7fa0c0134a50 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f803f8d9ba9cac server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:33.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.508+0000 7fa0c5138640 1 -- 172.21.15.60:0/623059670 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa0b0009dd0 con 0x7fa0c0132630 2023-08-24T10:07:33.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.508+0000 7fa0c5138640 1 -- 172.21.15.60:0/623059670 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa0b000e230 con 0x7fa0c0132630 2023-08-24T10:07:33.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.508+0000 7fa0c5138640 1 -- 172.21.15.60:0/623059670 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa0b000e4d0 con 0x7fa0c0132630 2023-08-24T10:07:33.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.509+0000 7fa0c839e640 1 -- 172.21.15.60:0/623059670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c0132630 msgr2=0x7fa0c0134a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:33.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.509+0000 7fa0c839e640 1 --2- 172.21.15.60:0/623059670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c0132630 0x7fa0c0134a50 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:33.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.509+0000 7fa0c839e640 1 -- 172.21.15.60:0/623059670 shutdown_connections 2023-08-24T10:07:33.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.509+0000 7fa0c839e640 1 --2- 172.21.15.60:0/623059670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c0132630 0x7fa0c0134a50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:33.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.509+0000 7fa0c839e640 1 -- 172.21.15.60:0/623059670 >> 172.21.15.60:0/623059670 conn(0x7fa0c009e110 msgr2=0x7fa0c009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:33.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.509+0000 7fa0c839e640 1 -- 172.21.15.60:0/623059670 shutdown_connections 2023-08-24T10:07:33.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.509+0000 7fa0c839e640 1 -- 172.21.15.60:0/623059670 wait complete. 2023-08-24T10:07:33.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.509+0000 7fa0c839e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:33.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.510+0000 7fa0c839e640 1 Processor -- start 2023-08-24T10:07:33.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.510+0000 7fa0c839e640 1 -- start start 2023-08-24T10:07:33.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.511+0000 7fa0c839e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c0096860 0x7fa0c006e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:33.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.511+0000 7fa0c839e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c0070410 con 0x7fa0c0096860 2023-08-24T10:07:33.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.511+0000 7fa0c613a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c0096860 0x7fa0c006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:33.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.511+0000 7fa0c613a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c0096860 0x7fa0c006e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40426/0 (socket says 172.21.15.60:40426) 2023-08-24T10:07:33.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.511+0000 7fa0c613a640 1 -- 172.21.15.60:0/2450338668 learned_addr learned my addr 172.21.15.60:0/2450338668 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:33.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.512+0000 7fa0c613a640 1 -- 172.21.15.60:0/2450338668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c0091fc0 con 0x7fa0c0096860 2023-08-24T10:07:33.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.512+0000 7fa0c613a640 1 --2- 172.21.15.60:0/2450338668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c0096860 0x7fa0c006e720 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:33.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.512+0000 7fa0b6ffd640 1 -- 172.21.15.60:0/2450338668 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa0bc00c7e0 con 0x7fa0c0096860 2023-08-24T10:07:33.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.512+0000 7fa0b6ffd640 1 -- 172.21.15.60:0/2450338668 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa0bc00c940 con 0x7fa0c0096860 2023-08-24T10:07:33.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.512+0000 7fa0c839e640 1 -- 172.21.15.60:0/2450338668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c0093080 con 0x7fa0c0096860 2023-08-24T10:07:33.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.512+0000 7fa0b6ffd640 1 -- 172.21.15.60:0/2450338668 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa0bc00cbe0 con 0x7fa0c0096860 2023-08-24T10:07:33.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.512+0000 7fa0c839e640 1 -- 172.21.15.60:0/2450338668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c0091920 con 0x7fa0c0096860 2023-08-24T10:07:33.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.513+0000 7fa0b6ffd640 1 -- 172.21.15.60:0/2450338668 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fa0bc011310 con 0x7fa0c0096860 2023-08-24T10:07:33.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.513+0000 7fa0b6ffd640 1 --2- 172.21.15.60:0/2450338668 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa0a4037130 0x7fa0a4039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:33.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.513+0000 7fa0b6ffd640 1 -- 172.21.15.60:0/2450338668 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fa0bc0469e0 con 0x7fa0c0096860 2023-08-24T10:07:33.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.513+0000 7fa0c839e640 1 -- 172.21.15.60:0/2450338668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c0061800 con 0x7fa0c0096860 2023-08-24T10:07:33.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.513+0000 7fa0c5939640 1 --2- 172.21.15.60:0/2450338668 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa0a4037130 0x7fa0a4039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:33.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.514+0000 7fa0c5939640 1 --2- 172.21.15.60:0/2450338668 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa0a4037130 0x7fa0a4039640 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:33.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.518+0000 7fa0b6ffd640 1 -- 172.21.15.60:0/2450338668 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fa0bc00ad00 con 0x7fa0c0096860 2023-08-24T10:07:33.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.692+0000 7fa0c839e640 1 -- 172.21.15.60:0/2450338668 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0c006b170 con 0x7fa0a4037130 2023-08-24T10:07:33.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.694+0000 7fa0b6ffd640 1 -- 172.21.15.60:0/2450338668 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7fa0b0009df0 con 0x7fa0a4037130 2023-08-24T10:07:33.695 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:33.695 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:33.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.695+0000 7fa0c839e640 1 -- 172.21.15.60:0/2450338668 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa0a4037130 msgr2=0x7fa0a4039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:33.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.695+0000 7fa0c839e640 1 --2- 172.21.15.60:0/2450338668 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa0a4037130 0x7fa0a4039640 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:33.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.695+0000 7fa0c839e640 1 -- 172.21.15.60:0/2450338668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c0096860 msgr2=0x7fa0c006e720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:33.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.695+0000 7fa0c839e640 1 --2- 172.21.15.60:0/2450338668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c0096860 0x7fa0c006e720 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:33.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.695+0000 7fa0c839e640 1 -- 172.21.15.60:0/2450338668 shutdown_connections 2023-08-24T10:07:33.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.695+0000 7fa0c839e640 1 --2- 172.21.15.60:0/2450338668 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa0a4037130 0x7fa0a4039640 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:33.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.695+0000 7fa0c839e640 1 --2- 172.21.15.60:0/2450338668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c0096860 0x7fa0c006e720 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:33.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.695+0000 7fa0c839e640 1 -- 172.21.15.60:0/2450338668 >> 172.21.15.60:0/2450338668 conn(0x7fa0c009e110 msgr2=0x7fa0c0131180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:33.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.695+0000 7fa0c839e640 1 -- 172.21.15.60:0/2450338668 shutdown_connections 2023-08-24T10:07:33.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.695+0000 7fa0c839e640 1 -- 172.21.15.60:0/2450338668 wait complete. 2023-08-24T10:07:33.710 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-08-24T10:07:33.525629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:33.712 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:33.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.819+0000 7fec95cac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:33.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.822+0000 7fec95cac640 1 Processor -- start 2023-08-24T10:07:33.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.823+0000 7fec95cac640 1 -- start start 2023-08-24T10:07:33.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.823+0000 7fec95cac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fec90135980 0x7fec90137da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:33.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.823+0000 7fec95cac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec90132610 con 0x7fec90132b50 2023-08-24T10:07:33.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.823+0000 7fec95cac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec9009c260 con 0x7fec90135980 2023-08-24T10:07:33.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.823+0000 7fec8effd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fec90135980 0x7fec90137da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:33.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.823+0000 7fec8effd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fec90135980 0x7fec90137da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40440/0 (socket says 172.21.15.60:40440) 2023-08-24T10:07:33.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.823+0000 7fec8effd640 1 -- 172.21.15.60:0/2376643746 learned_addr learned my addr 172.21.15.60:0/2376643746 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:33.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.824+0000 7fec8effd640 1 -- 172.21.15.60:0/2376643746 >> v1:172.21.15.60:6789/0 conn(0x7fec90132b50 legacy=0x7fec90132f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:33.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.824+0000 7fec8effd640 1 -- 172.21.15.60:0/2376643746 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec90132840 con 0x7fec90135980 2023-08-24T10:07:33.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.824+0000 7fec8effd640 1 --2- 172.21.15.60:0/2376643746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec90135980 0x7fec90137da0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d2fef1657403e5 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:33.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.824+0000 7fec8e7fc640 1 -- 172.21.15.60:0/2376643746 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fec84009dd0 con 0x7fec90135980 2023-08-24T10:07:33.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.824+0000 7fec8e7fc640 1 -- 172.21.15.60:0/2376643746 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fec8400e230 con 0x7fec90135980 2023-08-24T10:07:33.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.824+0000 7fec8e7fc640 1 -- 172.21.15.60:0/2376643746 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fec8400e4d0 con 0x7fec90135980 2023-08-24T10:07:33.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.825+0000 7fec95cac640 1 -- 172.21.15.60:0/2376643746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec90135980 msgr2=0x7fec90137da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:33.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.825+0000 7fec95cac640 1 --2- 172.21.15.60:0/2376643746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec90135980 0x7fec90137da0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:33.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.825+0000 7fec95cac640 1 -- 172.21.15.60:0/2376643746 shutdown_connections 2023-08-24T10:07:33.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.825+0000 7fec95cac640 1 --2- 172.21.15.60:0/2376643746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec90135980 0x7fec90137da0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:33.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.825+0000 7fec95cac640 1 -- 172.21.15.60:0/2376643746 >> 172.21.15.60:0/2376643746 conn(0x7fec900a26a0 msgr2=0x7fec900a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:33.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.825+0000 7fec95cac640 1 -- 172.21.15.60:0/2376643746 shutdown_connections 2023-08-24T10:07:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.825+0000 7fec95cac640 1 -- 172.21.15.60:0/2376643746 wait complete. 2023-08-24T10:07:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.825+0000 7fec95cac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.826+0000 7fec95cac640 1 Processor -- start 2023-08-24T10:07:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.826+0000 7fec95cac640 1 -- start start 2023-08-24T10:07:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.827+0000 7fec95cac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec90132b50 0x7fec900a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.827+0000 7fec95cac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec900a1b60 con 0x7fec90132b50 2023-08-24T10:07:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.827+0000 7fec8f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec90132b50 0x7fec900a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.827+0000 7fec8f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec90132b50 0x7fec900a0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40452/0 (socket says 172.21.15.60:40452) 2023-08-24T10:07:33.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.827+0000 7fec8f7fe640 1 -- 172.21.15.60:0/2469398871 learned_addr learned my addr 172.21.15.60:0/2469398871 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:33.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.827+0000 7fec8f7fe640 1 -- 172.21.15.60:0/2469398871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9009fab0 con 0x7fec90132b50 2023-08-24T10:07:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.827+0000 7fec8f7fe640 1 --2- 172.21.15.60:0/2469398871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec90132b50 0x7fec900a0000 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.827+0000 7fec77fff640 1 -- 172.21.15.60:0/2469398871 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fec8000c830 con 0x7fec90132b50 2023-08-24T10:07:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.827+0000 7fec77fff640 1 -- 172.21.15.60:0/2469398871 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fec8000c990 con 0x7fec90132b50 2023-08-24T10:07:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.827+0000 7fec95cac640 1 -- 172.21.15.60:0/2469398871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9006a240 con 0x7fec90132b50 2023-08-24T10:07:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.827+0000 7fec77fff640 1 -- 172.21.15.60:0/2469398871 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fec8000cc30 con 0x7fec90132b50 2023-08-24T10:07:33.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.827+0000 7fec95cac640 1 -- 172.21.15.60:0/2469398871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9011ac50 con 0x7fec90132b50 2023-08-24T10:07:33.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.828+0000 7fec77fff640 1 -- 172.21.15.60:0/2469398871 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fec80011310 con 0x7fec90132b50 2023-08-24T10:07:33.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.828+0000 7fec95cac640 1 -- 172.21.15.60:0/2469398871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec90061800 con 0x7fec90132b50 2023-08-24T10:07:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.828+0000 7fec77fff640 1 --2- 172.21.15.60:0/2469398871 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fec7c037130 0x7fec7c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.828+0000 7fec77fff640 1 -- 172.21.15.60:0/2469398871 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fec800467f0 con 0x7fec90132b50 2023-08-24T10:07:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.829+0000 7fec8effd640 1 --2- 172.21.15.60:0/2469398871 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fec7c037130 0x7fec7c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.829+0000 7fec8effd640 1 --2- 172.21.15.60:0/2469398871 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fec7c037130 0x7fec7c039640 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:33.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:33.833+0000 7fec77fff640 1 -- 172.21.15.60:0/2469398871 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fec8000a9a0 con 0x7fec90132b50 2023-08-24T10:07:34.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.004+0000 7fec95cac640 1 -- 172.21.15.60:0/2469398871 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec9009f940 con 0x7fec7c037130 2023-08-24T10:07:34.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.006+0000 7fec77fff640 1 -- 172.21.15.60:0/2469398871 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7fec9009f940 con 0x7fec7c037130 2023-08-24T10:07:34.007 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:34.008 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:34.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.008+0000 7fec95cac640 1 -- 172.21.15.60:0/2469398871 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fec7c037130 msgr2=0x7fec7c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:34.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.008+0000 7fec95cac640 1 --2- 172.21.15.60:0/2469398871 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fec7c037130 0x7fec7c039640 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:34.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.008+0000 7fec95cac640 1 -- 172.21.15.60:0/2469398871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec90132b50 msgr2=0x7fec900a0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:34.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.008+0000 7fec95cac640 1 --2- 172.21.15.60:0/2469398871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec90132b50 0x7fec900a0000 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:34.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.008+0000 7fec95cac640 1 -- 172.21.15.60:0/2469398871 shutdown_connections 2023-08-24T10:07:34.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.008+0000 7fec95cac640 1 --2- 172.21.15.60:0/2469398871 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fec7c037130 0x7fec7c039640 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:34.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.008+0000 7fec95cac640 1 --2- 172.21.15.60:0/2469398871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec90132b50 0x7fec900a0000 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:34.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.008+0000 7fec95cac640 1 -- 172.21.15.60:0/2469398871 >> 172.21.15.60:0/2469398871 conn(0x7fec900a26a0 msgr2=0x7fec901baaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:34.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.008+0000 7fec95cac640 1 -- 172.21.15.60:0/2469398871 shutdown_connections 2023-08-24T10:07:34.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.008+0000 7fec95cac640 1 -- 172.21.15.60:0/2469398871 wait complete. 2023-08-24T10:07:34.019 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-08-24T10:07:33.525629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:34.021 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:07:34.021 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:34.021 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:07:34.021 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:34.022 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:07:34.022 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:34.022 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:07:34.022 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:34.023 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:07:34.023 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:34.023 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:07:34.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:34.024 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:07:34.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:34.024 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:07:34.025 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:34.025 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:07:34.025 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:34.026 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:07:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.132+0000 7fe13bcac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:34.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.135+0000 7fe13bcac640 1 Processor -- start 2023-08-24T10:07:34.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.135+0000 7fe13bcac640 1 -- start start 2023-08-24T10:07:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.136+0000 7fe13bcac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe134132680 0x7fe134134aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.136+0000 7fe13bcac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe13406dfc0 con 0x7fe134070c30 2023-08-24T10:07:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.136+0000 7fe13bcac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe13406d9b0 con 0x7fe134132680 2023-08-24T10:07:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.136+0000 7fe139247640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe134132680 0x7fe134134aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.136+0000 7fe139247640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe134132680 0x7fe134134aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40468/0 (socket says 172.21.15.60:40468) 2023-08-24T10:07:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.136+0000 7fe139247640 1 -- 172.21.15.60:0/3041449599 learned_addr learned my addr 172.21.15.60:0/3041449599 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.136+0000 7fe139247640 1 -- 172.21.15.60:0/3041449599 >> v1:172.21.15.60:6789/0 conn(0x7fe134070c30 legacy=0x7fe134071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.136+0000 7fe139247640 1 -- 172.21.15.60:0/3041449599 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13406e6e0 con 0x7fe134132680 2023-08-24T10:07:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.137+0000 7fe139247640 1 --2- 172.21.15.60:0/3041449599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe134132680 0x7fe134134aa0 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3062e4a19795701d server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.137+0000 7fe138a46640 1 -- 172.21.15.60:0/3041449599 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe124009dd0 con 0x7fe134132680 2023-08-24T10:07:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.137+0000 7fe138a46640 1 -- 172.21.15.60:0/3041449599 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fe12400e230 con 0x7fe134132680 2023-08-24T10:07:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.137+0000 7fe138a46640 1 -- 172.21.15.60:0/3041449599 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe12400e4d0 con 0x7fe134132680 2023-08-24T10:07:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.137+0000 7fe13bcac640 1 -- 172.21.15.60:0/3041449599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe134132680 msgr2=0x7fe134134aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.137+0000 7fe13bcac640 1 --2- 172.21.15.60:0/3041449599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe134132680 0x7fe134134aa0 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.137+0000 7fe13bcac640 1 -- 172.21.15.60:0/3041449599 shutdown_connections 2023-08-24T10:07:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.137+0000 7fe13bcac640 1 --2- 172.21.15.60:0/3041449599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe134132680 0x7fe134134aa0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:34.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.137+0000 7fe13bcac640 1 -- 172.21.15.60:0/3041449599 >> 172.21.15.60:0/3041449599 conn(0x7fe13409e110 msgr2=0x7fe13409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:34.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.138+0000 7fe13bcac640 1 -- 172.21.15.60:0/3041449599 shutdown_connections 2023-08-24T10:07:34.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.138+0000 7fe13bcac640 1 -- 172.21.15.60:0/3041449599 wait complete. 2023-08-24T10:07:34.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.138+0000 7fe13bcac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:34.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.139+0000 7fe13bcac640 1 Processor -- start 2023-08-24T10:07:34.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.139+0000 7fe13bcac640 1 -- start start 2023-08-24T10:07:34.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.139+0000 7fe13bcac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe134070c30 0x7fe13412c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:34.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.139+0000 7fe13bcac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe134096cd0 con 0x7fe134070c30 2023-08-24T10:07:34.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.139+0000 7fe139a48640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe134070c30 0x7fe13412c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:34.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.139+0000 7fe139a48640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe134070c30 0x7fe13412c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40474/0 (socket says 172.21.15.60:40474) 2023-08-24T10:07:34.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.139+0000 7fe139a48640 1 -- 172.21.15.60:0/1987568372 learned_addr learned my addr 172.21.15.60:0/1987568372 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:34.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.140+0000 7fe139a48640 1 -- 172.21.15.60:0/1987568372 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe134096f50 con 0x7fe134070c30 2023-08-24T10:07:34.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.140+0000 7fe139a48640 1 --2- 172.21.15.60:0/1987568372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe134070c30 0x7fe13412c090 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:34.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.140+0000 7fe12a7fc640 1 -- 172.21.15.60:0/1987568372 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe13000c7e0 con 0x7fe134070c30 2023-08-24T10:07:34.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.140+0000 7fe12a7fc640 1 -- 172.21.15.60:0/1987568372 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fe13000c940 con 0x7fe134070c30 2023-08-24T10:07:34.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.140+0000 7fe12a7fc640 1 -- 172.21.15.60:0/1987568372 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe13000cbe0 con 0x7fe134070c30 2023-08-24T10:07:34.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.140+0000 7fe13bcac640 1 -- 172.21.15.60:0/1987568372 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13412ec30 con 0x7fe134070c30 2023-08-24T10:07:34.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.140+0000 7fe13bcac640 1 -- 172.21.15.60:0/1987568372 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13412e5b0 con 0x7fe134070c30 2023-08-24T10:07:34.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.141+0000 7fe12a7fc640 1 -- 172.21.15.60:0/1987568372 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fe130011310 con 0x7fe134070c30 2023-08-24T10:07:34.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.141+0000 7fe13bcac640 1 -- 172.21.15.60:0/1987568372 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe13412dc60 con 0x7fe134070c30 2023-08-24T10:07:34.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.141+0000 7fe12a7fc640 1 --2- 172.21.15.60:0/1987568372 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe118037130 0x7fe118039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:34.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.141+0000 7fe12a7fc640 1 -- 172.21.15.60:0/1987568372 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fe130046b00 con 0x7fe134070c30 2023-08-24T10:07:34.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.141+0000 7fe139247640 1 --2- 172.21.15.60:0/1987568372 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe118037130 0x7fe118039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:34.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.142+0000 7fe139247640 1 --2- 172.21.15.60:0/1987568372 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe118037130 0x7fe118039640 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:34.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.146+0000 7fe12a7fc640 1 -- 172.21.15.60:0/1987568372 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fe13000ad00 con 0x7fe134070c30 2023-08-24T10:07:34.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.388+0000 7fe13bcac640 1 -- 172.21.15.60:0/1987568372 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe134061800 con 0x7fe134070c30 2023-08-24T10:07:34.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.388+0000 7fe12a7fc640 1 -- 172.21.15.60:0/1987568372 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7fe13000a910 con 0x7fe134070c30 2023-08-24T10:07:34.390 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:34.391 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":165,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:07:34.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.390+0000 7fe13bcac640 1 -- 172.21.15.60:0/1987568372 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe118037130 msgr2=0x7fe118039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:34.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.390+0000 7fe13bcac640 1 --2- 172.21.15.60:0/1987568372 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe118037130 0x7fe118039640 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:34.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.390+0000 7fe13bcac640 1 -- 172.21.15.60:0/1987568372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe134070c30 msgr2=0x7fe13412c090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:34.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.390+0000 7fe13bcac640 1 --2- 172.21.15.60:0/1987568372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe134070c30 0x7fe13412c090 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:34.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.390+0000 7fe13bcac640 1 -- 172.21.15.60:0/1987568372 shutdown_connections 2023-08-24T10:07:34.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.390+0000 7fe13bcac640 1 --2- 172.21.15.60:0/1987568372 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe118037130 0x7fe118039640 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:34.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.390+0000 7fe13bcac640 1 --2- 172.21.15.60:0/1987568372 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe134070c30 0x7fe13412c090 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:34.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.390+0000 7fe13bcac640 1 -- 172.21.15.60:0/1987568372 >> 172.21.15.60:0/1987568372 conn(0x7fe13409e110 msgr2=0x7fe13409ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:34.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.390+0000 7fe13bcac640 1 -- 172.21.15.60:0/1987568372 shutdown_connections 2023-08-24T10:07:34.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.390+0000 7fe13bcac640 1 -- 172.21.15.60:0/1987568372 wait complete. 2023-08-24T10:07:34.404 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:07:34.404 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:34.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.511+0000 7fd2b9027640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:34.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.515+0000 7fd2b9027640 1 Processor -- start 2023-08-24T10:07:34.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.515+0000 7fd2b9027640 1 -- start start 2023-08-24T10:07:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.516+0000 7fd2b9027640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd2b4071e40 0x7fd2b4096800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.516+0000 7fd2b9027640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b40977f0 con 0x7fd2b406f100 2023-08-24T10:07:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.516+0000 7fd2b9027640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b409a280 con 0x7fd2b4071e40 2023-08-24T10:07:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.516+0000 7fd2b2d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd2b4071e40 0x7fd2b4096800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.516+0000 7fd2b2d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd2b4071e40 0x7fd2b4096800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40482/0 (socket says 172.21.15.60:40482) 2023-08-24T10:07:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.516+0000 7fd2b2d9d640 1 -- 172.21.15.60:0/2400819480 learned_addr learned my addr 172.21.15.60:0/2400819480 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.516+0000 7fd2b2d9d640 1 -- 172.21.15.60:0/2400819480 >> v1:172.21.15.60:6789/0 conn(0x7fd2b406f100 legacy=0x7fd2b406f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.516+0000 7fd2b2d9d640 1 -- 172.21.15.60:0/2400819480 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b409a6b0 con 0x7fd2b4071e40 2023-08-24T10:07:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.516+0000 7fd2b2d9d640 1 --2- 172.21.15.60:0/2400819480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2b4071e40 0x7fd2b4096800 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502c0fdfb8dd4df0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.516+0000 7fd2b259c640 1 -- 172.21.15.60:0/2400819480 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd2a4009dd0 con 0x7fd2b4071e40 2023-08-24T10:07:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.516+0000 7fd2b259c640 1 -- 172.21.15.60:0/2400819480 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd2a400e230 con 0x7fd2b4071e40 2023-08-24T10:07:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.516+0000 7fd2b259c640 1 -- 172.21.15.60:0/2400819480 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd2a400e4d0 con 0x7fd2b4071e40 2023-08-24T10:07:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.517+0000 7fd2b9027640 1 -- 172.21.15.60:0/2400819480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2b4071e40 msgr2=0x7fd2b4096800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.517+0000 7fd2b9027640 1 --2- 172.21.15.60:0/2400819480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2b4071e40 0x7fd2b4096800 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.517+0000 7fd2b9027640 1 -- 172.21.15.60:0/2400819480 shutdown_connections 2023-08-24T10:07:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.517+0000 7fd2b9027640 1 --2- 172.21.15.60:0/2400819480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2b4071e40 0x7fd2b4096800 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.517+0000 7fd2b9027640 1 -- 172.21.15.60:0/2400819480 >> 172.21.15.60:0/2400819480 conn(0x7fd2b409e110 msgr2=0x7fd2b409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.517+0000 7fd2b9027640 1 -- 172.21.15.60:0/2400819480 shutdown_connections 2023-08-24T10:07:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.517+0000 7fd2b9027640 1 -- 172.21.15.60:0/2400819480 wait complete. 2023-08-24T10:07:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.517+0000 7fd2b9027640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.518+0000 7fd2b9027640 1 Processor -- start 2023-08-24T10:07:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.519+0000 7fd2b9027640 1 -- start start 2023-08-24T10:07:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.519+0000 7fd2b9027640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2b406f100 0x7fd2b41c19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.519+0000 7fd2b9027640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b408ffe0 con 0x7fd2b406f100 2023-08-24T10:07:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.519+0000 7fd2b2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2b406f100 0x7fd2b41c19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.519+0000 7fd2b2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2b406f100 0x7fd2b41c19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40488/0 (socket says 172.21.15.60:40488) 2023-08-24T10:07:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.519+0000 7fd2b2d9d640 1 -- 172.21.15.60:0/4066071517 learned_addr learned my addr 172.21.15.60:0/4066071517 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.520+0000 7fd2b2d9d640 1 -- 172.21.15.60:0/4066071517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b41c14b0 con 0x7fd2b406f100 2023-08-24T10:07:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.520+0000 7fd2b2d9d640 1 --2- 172.21.15.60:0/4066071517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2b406f100 0x7fd2b41c19f0 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.520+0000 7fd2ab7fe640 1 -- 172.21.15.60:0/4066071517 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd2a4014540 con 0x7fd2b406f100 2023-08-24T10:07:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.520+0000 7fd2ab7fe640 1 -- 172.21.15.60:0/4066071517 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd2a40146a0 con 0x7fd2b406f100 2023-08-24T10:07:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.520+0000 7fd2b9027640 1 -- 172.21.15.60:0/4066071517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b4090720 con 0x7fd2b406f100 2023-08-24T10:07:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.520+0000 7fd2ab7fe640 1 -- 172.21.15.60:0/4066071517 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd2a4014940 con 0x7fd2b406f100 2023-08-24T10:07:34.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.520+0000 7fd2b9027640 1 -- 172.21.15.60:0/4066071517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b4092c10 con 0x7fd2b406f100 2023-08-24T10:07:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.521+0000 7fd2ab7fe640 1 -- 172.21.15.60:0/4066071517 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fd2a4014af0 con 0x7fd2b406f100 2023-08-24T10:07:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.521+0000 7fd2b9027640 1 -- 172.21.15.60:0/4066071517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b4061800 con 0x7fd2b406f100 2023-08-24T10:07:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.521+0000 7fd2ab7fe640 1 --2- 172.21.15.60:0/4066071517 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd294037130 0x7fd294039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.521+0000 7fd2ab7fe640 1 -- 172.21.15.60:0/4066071517 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fd2a4019040 con 0x7fd2b406f100 2023-08-24T10:07:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.521+0000 7fd2abfff640 1 --2- 172.21.15.60:0/4066071517 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd294037130 0x7fd294039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.522+0000 7fd2abfff640 1 --2- 172.21.15.60:0/4066071517 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd294037130 0x7fd294039640 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:34.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.526+0000 7fd2ab7fe640 1 -- 172.21.15.60:0/4066071517 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fd2a4018190 con 0x7fd2b406f100 2023-08-24T10:07:34.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.696+0000 7fd2b9027640 1 -- 172.21.15.60:0/4066071517 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd2b4097a80 con 0x7fd294037130 2023-08-24T10:07:34.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.697+0000 7fd2ab7fe640 1 -- 172.21.15.60:0/4066071517 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7fd29c001650 con 0x7fd294037130 2023-08-24T10:07:34.700 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:34.701 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:34.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.700+0000 7fd2b9027640 1 -- 172.21.15.60:0/4066071517 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd294037130 msgr2=0x7fd294039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:34.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.700+0000 7fd2b9027640 1 --2- 172.21.15.60:0/4066071517 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd294037130 0x7fd294039640 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:34.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.700+0000 7fd2b9027640 1 -- 172.21.15.60:0/4066071517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2b406f100 msgr2=0x7fd2b41c19f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:34.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.700+0000 7fd2b9027640 1 --2- 172.21.15.60:0/4066071517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2b406f100 0x7fd2b41c19f0 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:34.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.700+0000 7fd2b9027640 1 -- 172.21.15.60:0/4066071517 shutdown_connections 2023-08-24T10:07:34.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.700+0000 7fd2b9027640 1 --2- 172.21.15.60:0/4066071517 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd294037130 0x7fd294039640 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:34.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.700+0000 7fd2b9027640 1 --2- 172.21.15.60:0/4066071517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2b406f100 0x7fd2b41c19f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:34.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.700+0000 7fd2b9027640 1 -- 172.21.15.60:0/4066071517 >> 172.21.15.60:0/4066071517 conn(0x7fd2b409e110 msgr2=0x7fd2b40985b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:34.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.700+0000 7fd2b9027640 1 -- 172.21.15.60:0/4066071517 shutdown_connections 2023-08-24T10:07:34.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:34.700+0000 7fd2b9027640 1 -- 172.21.15.60:0/4066071517 wait complete. 2023-08-24T10:07:34.711 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-08-24T10:07:33.525629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:37.713 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:37.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.833+0000 7ff966aec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:37.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.838+0000 7ff966aec640 1 Processor -- start 2023-08-24T10:07:37.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.838+0000 7ff966aec640 1 -- start start 2023-08-24T10:07:37.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.838+0000 7ff966aec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff960132b50 0x7ff960132f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:37.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.838+0000 7ff966aec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff960132610 con 0x7ff96012f110 2023-08-24T10:07:37.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.838+0000 7ff966aec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff96009c260 con 0x7ff960132b50 2023-08-24T10:07:37.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.839+0000 7ff964888640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff960132b50 0x7ff960132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:37.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.839+0000 7ff964888640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff960132b50 0x7ff960132f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50058/0 (socket says 172.21.15.60:50058) 2023-08-24T10:07:37.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.839+0000 7ff95ffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff96012f110 0x7ff96012f520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43592/0 (socket says 172.21.15.60:43592) 2023-08-24T10:07:37.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.839+0000 7ff964888640 1 -- 172.21.15.60:0/595563845 learned_addr learned my addr 172.21.15.60:0/595563845 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:37.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.839+0000 7ff964888640 1 -- 172.21.15.60:0/595563845 >> v1:172.21.15.60:6789/0 conn(0x7ff96012f110 legacy=0x7ff96012f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:37.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.839+0000 7ff964888640 1 -- 172.21.15.60:0/595563845 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff960132840 con 0x7ff960132b50 2023-08-24T10:07:37.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.839+0000 7ff964888640 1 --2- 172.21.15.60:0/595563845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff960132b50 0x7ff960132f60 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974f02622b117eb9 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:37.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.839+0000 7ff95f7fe640 1 -- 172.21.15.60:0/595563845 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff954009dd0 con 0x7ff960132b50 2023-08-24T10:07:37.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.839+0000 7ff95f7fe640 1 -- 172.21.15.60:0/595563845 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ff95400e230 con 0x7ff960132b50 2023-08-24T10:07:37.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.839+0000 7ff95f7fe640 1 -- 172.21.15.60:0/595563845 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff95400e4d0 con 0x7ff960132b50 2023-08-24T10:07:37.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.840+0000 7ff966aec640 1 -- 172.21.15.60:0/595563845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff960132b50 msgr2=0x7ff960132f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:37.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.840+0000 7ff966aec640 1 --2- 172.21.15.60:0/595563845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff960132b50 0x7ff960132f60 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:37.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.840+0000 7ff966aec640 1 -- 172.21.15.60:0/595563845 shutdown_connections 2023-08-24T10:07:37.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.840+0000 7ff966aec640 1 --2- 172.21.15.60:0/595563845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff960132b50 0x7ff960132f60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:37.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.840+0000 7ff966aec640 1 -- 172.21.15.60:0/595563845 >> 172.21.15.60:0/595563845 conn(0x7ff9600a26a0 msgr2=0x7ff9600a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:37.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.840+0000 7ff966aec640 1 -- 172.21.15.60:0/595563845 shutdown_connections 2023-08-24T10:07:37.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.840+0000 7ff966aec640 1 -- 172.21.15.60:0/595563845 wait complete. 2023-08-24T10:07:37.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.840+0000 7ff966aec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:37.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.841+0000 7ff966aec640 1 Processor -- start 2023-08-24T10:07:37.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.841+0000 7ff966aec640 1 -- start start 2023-08-24T10:07:37.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.842+0000 7ff966aec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff96012f110 0x7ff96009fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:37.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.842+0000 7ff966aec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96012cc80 con 0x7ff96012f110 2023-08-24T10:07:37.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.842+0000 7ff964888640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff96012f110 0x7ff96009fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:37.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.842+0000 7ff964888640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff96012f110 0x7ff96009fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50072/0 (socket says 172.21.15.60:50072) 2023-08-24T10:07:37.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.842+0000 7ff964888640 1 -- 172.21.15.60:0/452928400 learned_addr learned my addr 172.21.15.60:0/452928400 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:37.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.842+0000 7ff964888640 1 -- 172.21.15.60:0/452928400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96009f400 con 0x7ff96012f110 2023-08-24T10:07:37.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.842+0000 7ff964888640 1 --2- 172.21.15.60:0/452928400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff96012f110 0x7ff96009fcd0 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:37.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.843+0000 7ff95d7fa640 1 -- 172.21.15.60:0/452928400 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff954014410 con 0x7ff96012f110 2023-08-24T10:07:37.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.843+0000 7ff95d7fa640 1 -- 172.21.15.60:0/452928400 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ff954014570 con 0x7ff96012f110 2023-08-24T10:07:37.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.843+0000 7ff966aec640 1 -- 172.21.15.60:0/452928400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9600a18e0 con 0x7ff96012f110 2023-08-24T10:07:37.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.843+0000 7ff95d7fa640 1 -- 172.21.15.60:0/452928400 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff954014810 con 0x7ff96012f110 2023-08-24T10:07:37.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.843+0000 7ff966aec640 1 -- 172.21.15.60:0/452928400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96006a1d0 con 0x7ff96012f110 2023-08-24T10:07:37.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.844+0000 7ff95d7fa640 1 -- 172.21.15.60:0/452928400 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7ff9540149c0 con 0x7ff96012f110 2023-08-24T10:07:37.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.844+0000 7ff966aec640 1 -- 172.21.15.60:0/452928400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff96011c2b0 con 0x7ff96012f110 2023-08-24T10:07:37.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.844+0000 7ff95d7fa640 1 --2- 172.21.15.60:0/452928400 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff944037130 0x7ff944039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:37.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.844+0000 7ff95d7fa640 1 -- 172.21.15.60:0/452928400 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7ff954019040 con 0x7ff96012f110 2023-08-24T10:07:37.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.844+0000 7ff95ffff640 1 --2- 172.21.15.60:0/452928400 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff944037130 0x7ff944039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:37.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.845+0000 7ff95ffff640 1 --2- 172.21.15.60:0/452928400 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff944037130 0x7ff944039640 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:37.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:37.849+0000 7ff95d7fa640 1 -- 172.21.15.60:0/452928400 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7ff95400f8f0 con 0x7ff96012f110 2023-08-24T10:07:38.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.022+0000 7ff966aec640 1 -- 172.21.15.60:0/452928400 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff9601334a0 con 0x7ff944037130 2023-08-24T10:07:38.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.024+0000 7ff95d7fa640 1 -- 172.21.15.60:0/452928400 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7ff94c0070b0 con 0x7ff944037130 2023-08-24T10:07:38.030 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:38.032 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:38.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.025+0000 7ff966aec640 1 -- 172.21.15.60:0/452928400 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff944037130 msgr2=0x7ff944039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:38.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.025+0000 7ff966aec640 1 --2- 172.21.15.60:0/452928400 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff944037130 0x7ff944039640 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:38.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.025+0000 7ff966aec640 1 -- 172.21.15.60:0/452928400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff96012f110 msgr2=0x7ff96009fcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:38.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.025+0000 7ff966aec640 1 --2- 172.21.15.60:0/452928400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff96012f110 0x7ff96009fcd0 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:38.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.025+0000 7ff966aec640 1 -- 172.21.15.60:0/452928400 shutdown_connections 2023-08-24T10:07:38.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.025+0000 7ff966aec640 1 --2- 172.21.15.60:0/452928400 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff944037130 0x7ff944039640 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:38.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.025+0000 7ff966aec640 1 --2- 172.21.15.60:0/452928400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff96012f110 0x7ff96009fcd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:38.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.025+0000 7ff966aec640 1 -- 172.21.15.60:0/452928400 >> 172.21.15.60:0/452928400 conn(0x7ff9600a26a0 msgr2=0x7ff9601ba920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:38.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.025+0000 7ff966aec640 1 -- 172.21.15.60:0/452928400 shutdown_connections 2023-08-24T10:07:38.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.025+0000 7ff966aec640 1 -- 172.21.15.60:0/452928400 wait complete. 2023-08-24T10:07:38.037 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2023-08-24T10:07:37.526467+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:38.038 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:38.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.145+0000 7f785af9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:38.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.149+0000 7f785af9e640 1 Processor -- start 2023-08-24T10:07:38.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.149+0000 7f785af9e640 1 -- start start 2023-08-24T10:07:38.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.150+0000 7f785af9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7854135980 0x7f7854137da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:38.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.150+0000 7f785af9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7854132610 con 0x7f7854132b50 2023-08-24T10:07:38.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.150+0000 7f785af9e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f785409c260 con 0x7f7854135980 2023-08-24T10:07:38.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.150+0000 7f7853fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7854135980 0x7f7854137da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:38.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.150+0000 7f7853fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7854135980 0x7f7854137da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50074/0 (socket says 172.21.15.60:50074) 2023-08-24T10:07:38.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.150+0000 7f7853fff640 1 -- 172.21.15.60:0/3532410203 learned_addr learned my addr 172.21.15.60:0/3532410203 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:38.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.150+0000 7f7858d3a640 1 --1- 172.21.15.60:0/3532410203 >> v1:172.21.15.60:6789/0 conn(0x7f7854132b50 0x7f7854132f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43594/0 (socket says 172.21.15.60:43594) 2023-08-24T10:07:38.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.151+0000 7f7853fff640 1 -- 172.21.15.60:0/3532410203 >> v1:172.21.15.60:6789/0 conn(0x7f7854132b50 legacy=0x7f7854132f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:38.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.151+0000 7f7853fff640 1 -- 172.21.15.60:0/3532410203 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7854132840 con 0x7f7854135980 2023-08-24T10:07:38.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.151+0000 7f7853fff640 1 --2- 172.21.15.60:0/3532410203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7854135980 0x7f7854137da0 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f101eb61a667b5 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:38.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.151+0000 7f78537fe640 1 -- 172.21.15.60:0/3532410203 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7848009dd0 con 0x7f7854135980 2023-08-24T10:07:38.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.151+0000 7f78537fe640 1 -- 172.21.15.60:0/3532410203 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f784800e230 con 0x7f7854135980 2023-08-24T10:07:38.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.151+0000 7f78537fe640 1 -- 172.21.15.60:0/3532410203 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f784800e4d0 con 0x7f7854135980 2023-08-24T10:07:38.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.151+0000 7f785af9e640 1 -- 172.21.15.60:0/3532410203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7854135980 msgr2=0x7f7854137da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:38.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.151+0000 7f785af9e640 1 --2- 172.21.15.60:0/3532410203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7854135980 0x7f7854137da0 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:38.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.152+0000 7f785af9e640 1 -- 172.21.15.60:0/3532410203 shutdown_connections 2023-08-24T10:07:38.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.152+0000 7f785af9e640 1 --2- 172.21.15.60:0/3532410203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7854135980 0x7f7854137da0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:38.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.152+0000 7f785af9e640 1 -- 172.21.15.60:0/3532410203 >> 172.21.15.60:0/3532410203 conn(0x7f78540a26a0 msgr2=0x7f78540a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:38.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.152+0000 7f785af9e640 1 -- 172.21.15.60:0/3532410203 shutdown_connections 2023-08-24T10:07:38.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.152+0000 7f785af9e640 1 -- 172.21.15.60:0/3532410203 wait complete. 2023-08-24T10:07:38.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.152+0000 7f785af9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:38.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.153+0000 7f785af9e640 1 Processor -- start 2023-08-24T10:07:38.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.153+0000 7f785af9e640 1 -- start start 2023-08-24T10:07:38.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.154+0000 7f785af9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7854132b50 0x7f78540a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:38.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.154+0000 7f785af9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78540a1b60 con 0x7f7854132b50 2023-08-24T10:07:38.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.154+0000 7f7858d3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7854132b50 0x7f78540a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:38.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.154+0000 7f7858d3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7854132b50 0x7f78540a0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50082/0 (socket says 172.21.15.60:50082) 2023-08-24T10:07:38.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.154+0000 7f7858d3a640 1 -- 172.21.15.60:0/635701893 learned_addr learned my addr 172.21.15.60:0/635701893 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:38.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.154+0000 7f7858d3a640 1 -- 172.21.15.60:0/635701893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785409fab0 con 0x7f7854132b50 2023-08-24T10:07:38.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.154+0000 7f7858d3a640 1 --2- 172.21.15.60:0/635701893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7854132b50 0x7f78540a0000 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:38.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.154+0000 7f78517fa640 1 -- 172.21.15.60:0/635701893 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f78440071e0 con 0x7f7854132b50 2023-08-24T10:07:38.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.154+0000 7f78517fa640 1 -- 172.21.15.60:0/635701893 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7844007340 con 0x7f7854132b50 2023-08-24T10:07:38.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.154+0000 7f785af9e640 1 -- 172.21.15.60:0/635701893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785406a240 con 0x7f7854132b50 2023-08-24T10:07:38.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.154+0000 7f78517fa640 1 -- 172.21.15.60:0/635701893 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f78440075e0 con 0x7f7854132b50 2023-08-24T10:07:38.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.154+0000 7f785af9e640 1 -- 172.21.15.60:0/635701893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785411ac50 con 0x7f7854132b50 2023-08-24T10:07:38.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.155+0000 7f785af9e640 1 -- 172.21.15.60:0/635701893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7854061800 con 0x7f7854132b50 2023-08-24T10:07:38.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.155+0000 7f78517fa640 1 -- 172.21.15.60:0/635701893 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f78440057e0 con 0x7f7854132b50 2023-08-24T10:07:38.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.156+0000 7f78517fa640 1 --2- 172.21.15.60:0/635701893 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7840037130 0x7f7840039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:38.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.156+0000 7f78517fa640 1 -- 172.21.15.60:0/635701893 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f7844046090 con 0x7f7854132b50 2023-08-24T10:07:38.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.156+0000 7f7853fff640 1 --2- 172.21.15.60:0/635701893 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7840037130 0x7f7840039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:38.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.156+0000 7f7853fff640 1 --2- 172.21.15.60:0/635701893 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7840037130 0x7f7840039640 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:38.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.160+0000 7f78517fa640 1 -- 172.21.15.60:0/635701893 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f784400d330 con 0x7f7854132b50 2023-08-24T10:07:38.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.333+0000 7f785af9e640 1 -- 172.21.15.60:0/635701893 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f785409f940 con 0x7f7840037130 2023-08-24T10:07:38.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.334+0000 7f78517fa640 1 -- 172.21.15.60:0/635701893 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f785409f940 con 0x7f7840037130 2023-08-24T10:07:38.336 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:38.336 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:38.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.336+0000 7f785af9e640 1 -- 172.21.15.60:0/635701893 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7840037130 msgr2=0x7f7840039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:38.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.336+0000 7f785af9e640 1 --2- 172.21.15.60:0/635701893 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7840037130 0x7f7840039640 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.336+0000 7f785af9e640 1 -- 172.21.15.60:0/635701893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7854132b50 msgr2=0x7f78540a0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.336+0000 7f785af9e640 1 --2- 172.21.15.60:0/635701893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7854132b50 0x7f78540a0000 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.336+0000 7f785af9e640 1 -- 172.21.15.60:0/635701893 shutdown_connections 2023-08-24T10:07:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.336+0000 7f785af9e640 1 --2- 172.21.15.60:0/635701893 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7840037130 0x7f7840039640 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.336+0000 7f785af9e640 1 --2- 172.21.15.60:0/635701893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7854132b50 0x7f78540a0000 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.336+0000 7f785af9e640 1 -- 172.21.15.60:0/635701893 >> 172.21.15.60:0/635701893 conn(0x7f78540a26a0 msgr2=0x7f78541baaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:38.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.336+0000 7f785af9e640 1 -- 172.21.15.60:0/635701893 shutdown_connections 2023-08-24T10:07:38.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.336+0000 7f785af9e640 1 -- 172.21.15.60:0/635701893 wait complete. 2023-08-24T10:07:38.348 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2023-08-24T10:07:37.526467+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:38.349 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:07:38.349 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:38.349 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:07:38.350 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:38.350 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:07:38.350 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:38.350 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:07:38.351 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:38.351 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:07:38.351 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:38.351 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:07:38.352 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:38.352 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:07:38.352 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:38.352 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:07:38.353 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:38.353 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:07:38.353 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:38.353 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:07:38.362 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~151s 2023-08-24T10:07:38.362 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~151s 2023-08-24T10:07:38.362 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~151s 2023-08-24T10:07:38.363 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~151s 2023-08-24T10:07:38.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.460+0000 7fbde1d9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:38.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.463+0000 7fbde1d9e640 1 Processor -- start 2023-08-24T10:07:38.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.463+0000 7fbde1d9e640 1 -- start start 2023-08-24T10:07:38.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.464+0000 7fbde1d9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbddc12f610 0x7fbddc12fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:38.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.464+0000 7fbde1d9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbddc09d750 con 0x7fbddc1326f0 2023-08-24T10:07:38.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.464+0000 7fbde1d9e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbddc09d980 con 0x7fbddc12f610 2023-08-24T10:07:38.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.464+0000 7fbddb7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbddc12f610 0x7fbddc12fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:38.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.464+0000 7fbddb7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbddc12f610 0x7fbddc12fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50086/0 (socket says 172.21.15.60:50086) 2023-08-24T10:07:38.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.464+0000 7fbddb7fe640 1 -- 172.21.15.60:0/3581650750 learned_addr learned my addr 172.21.15.60:0/3581650750 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:38.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.464+0000 7fbddaffd640 1 --1- 172.21.15.60:0/3581650750 >> v1:172.21.15.60:6789/0 conn(0x7fbddc1326f0 0x7fbddc132b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43610/0 (socket says 172.21.15.60:43610) 2023-08-24T10:07:38.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.465+0000 7fbddb7fe640 1 -- 172.21.15.60:0/3581650750 >> v1:172.21.15.60:6789/0 conn(0x7fbddc1326f0 legacy=0x7fbddc132b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:38.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.465+0000 7fbddb7fe640 1 -- 172.21.15.60:0/3581650750 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc09bde0 con 0x7fbddc12f610 2023-08-24T10:07:38.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.465+0000 7fbddb7fe640 1 --2- 172.21.15.60:0/3581650750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc12f610 0x7fbddc12fa20 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3261e73ea9d408c server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:38.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.465+0000 7fbdda7fc640 1 -- 172.21.15.60:0/3581650750 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbdcc009dc0 con 0x7fbddc12f610 2023-08-24T10:07:38.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.465+0000 7fbdda7fc640 1 -- 172.21.15.60:0/3581650750 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fbdcc00e230 con 0x7fbddc12f610 2023-08-24T10:07:38.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.465+0000 7fbdda7fc640 1 -- 172.21.15.60:0/3581650750 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbdcc00e4d0 con 0x7fbddc12f610 2023-08-24T10:07:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.465+0000 7fbde1d9e640 1 -- 172.21.15.60:0/3581650750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc12f610 msgr2=0x7fbddc12fa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.465+0000 7fbde1d9e640 1 --2- 172.21.15.60:0/3581650750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc12f610 0x7fbddc12fa20 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.466+0000 7fbde1d9e640 1 -- 172.21.15.60:0/3581650750 shutdown_connections 2023-08-24T10:07:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.466+0000 7fbde1d9e640 1 --2- 172.21.15.60:0/3581650750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc12f610 0x7fbddc12fa20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.466+0000 7fbde1d9e640 1 -- 172.21.15.60:0/3581650750 >> 172.21.15.60:0/3581650750 conn(0x7fbddc0a2500 msgr2=0x7fbddc0a2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.466+0000 7fbde1d9e640 1 -- 172.21.15.60:0/3581650750 shutdown_connections 2023-08-24T10:07:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.466+0000 7fbde1d9e640 1 -- 172.21.15.60:0/3581650750 wait complete. 2023-08-24T10:07:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.466+0000 7fbde1d9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.467+0000 7fbde1d9e640 1 Processor -- start 2023-08-24T10:07:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.467+0000 7fbde1d9e640 1 -- start start 2023-08-24T10:07:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.468+0000 7fbde1d9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc12f610 0x7fbddc09fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.468+0000 7fbde1d9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc12cc60 con 0x7fbddc12f610 2023-08-24T10:07:38.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.468+0000 7fbddb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc12f610 0x7fbddc09fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:38.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.468+0000 7fbddb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc12f610 0x7fbddc09fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50100/0 (socket says 172.21.15.60:50100) 2023-08-24T10:07:38.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.468+0000 7fbddb7fe640 1 -- 172.21.15.60:0/690748863 learned_addr learned my addr 172.21.15.60:0/690748863 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:38.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.468+0000 7fbddb7fe640 1 -- 172.21.15.60:0/690748863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc06a1f0 con 0x7fbddc12f610 2023-08-24T10:07:38.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.468+0000 7fbddb7fe640 1 --2- 172.21.15.60:0/690748863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc12f610 0x7fbddc09fca0 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:38.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.468+0000 7fbdc3fff640 1 -- 172.21.15.60:0/690748863 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbdcc014540 con 0x7fbddc12f610 2023-08-24T10:07:38.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.468+0000 7fbdc3fff640 1 -- 172.21.15.60:0/690748863 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fbdcc0146a0 con 0x7fbddc12f610 2023-08-24T10:07:38.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.468+0000 7fbde1d9e640 1 -- 172.21.15.60:0/690748863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbddc09f7e0 con 0x7fbddc12f610 2023-08-24T10:07:38.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.468+0000 7fbdc3fff640 1 -- 172.21.15.60:0/690748863 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbdcc014940 con 0x7fbddc12f610 2023-08-24T10:07:38.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.468+0000 7fbde1d9e640 1 -- 172.21.15.60:0/690748863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbddc09f4d0 con 0x7fbddc12f610 2023-08-24T10:07:38.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.469+0000 7fbdc3fff640 1 -- 172.21.15.60:0/690748863 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fbdcc014af0 con 0x7fbddc12f610 2023-08-24T10:07:38.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.469+0000 7fbde1d9e640 1 -- 172.21.15.60:0/690748863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbddc061800 con 0x7fbddc12f610 2023-08-24T10:07:38.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.469+0000 7fbdc3fff640 1 --2- 172.21.15.60:0/690748863 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fbdc8037130 0x7fbdc8039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:38.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.469+0000 7fbdc3fff640 1 -- 172.21.15.60:0/690748863 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fbdcc019040 con 0x7fbddc12f610 2023-08-24T10:07:38.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.470+0000 7fbddaffd640 1 --2- 172.21.15.60:0/690748863 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fbdc8037130 0x7fbdc8039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:38.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.470+0000 7fbddaffd640 1 --2- 172.21.15.60:0/690748863 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fbdc8037130 0x7fbdc8039640 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:38.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.474+0000 7fbdc3fff640 1 -- 172.21.15.60:0/690748863 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fbdcc00fb00 con 0x7fbddc12f610 2023-08-24T10:07:38.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.718+0000 7fbde1d9e640 1 -- 172.21.15.60:0/690748863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fbddc06a1f0 con 0x7fbddc12f610 2023-08-24T10:07:38.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.718+0000 7fbdc3fff640 1 -- 172.21.15.60:0/690748863 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7fbddc06a1f0 con 0x7fbddc12f610 2023-08-24T10:07:38.720 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:38.721 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":169,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:07:38.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.720+0000 7fbde1d9e640 1 -- 172.21.15.60:0/690748863 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fbdc8037130 msgr2=0x7fbdc8039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:38.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.720+0000 7fbde1d9e640 1 --2- 172.21.15.60:0/690748863 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fbdc8037130 0x7fbdc8039640 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:38.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.720+0000 7fbde1d9e640 1 -- 172.21.15.60:0/690748863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc12f610 msgr2=0x7fbddc09fca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:38.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.720+0000 7fbde1d9e640 1 --2- 172.21.15.60:0/690748863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc12f610 0x7fbddc09fca0 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:38.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.720+0000 7fbde1d9e640 1 -- 172.21.15.60:0/690748863 shutdown_connections 2023-08-24T10:07:38.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.720+0000 7fbde1d9e640 1 --2- 172.21.15.60:0/690748863 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fbdc8037130 0x7fbdc8039640 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:38.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.720+0000 7fbde1d9e640 1 --2- 172.21.15.60:0/690748863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbddc12f610 0x7fbddc09fca0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:38.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.720+0000 7fbde1d9e640 1 -- 172.21.15.60:0/690748863 >> 172.21.15.60:0/690748863 conn(0x7fbddc0a2500 msgr2=0x7fbddc1ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:38.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.720+0000 7fbde1d9e640 1 -- 172.21.15.60:0/690748863 shutdown_connections 2023-08-24T10:07:38.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.720+0000 7fbde1d9e640 1 -- 172.21.15.60:0/690748863 wait complete. 2023-08-24T10:07:38.732 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:07:38.733 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:38.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.840+0000 7f69800ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:38.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.843+0000 7f69800ac640 1 Processor -- start 2023-08-24T10:07:38.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.844+0000 7f69800ac640 1 -- start start 2023-08-24T10:07:38.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.844+0000 7f69800ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6978135980 0x7f6978137da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:38.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.844+0000 7f69800ac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6978132610 con 0x7f6978132b50 2023-08-24T10:07:38.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.844+0000 7f69800ac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f697809c260 con 0x7f6978135980 2023-08-24T10:07:38.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.844+0000 7f697d647640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6978135980 0x7f6978137da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:38.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.845+0000 7f697d647640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6978135980 0x7f6978137da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50104/0 (socket says 172.21.15.60:50104) 2023-08-24T10:07:38.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.845+0000 7f697d647640 1 -- 172.21.15.60:0/370371625 learned_addr learned my addr 172.21.15.60:0/370371625 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:38.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.845+0000 7f697de48640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6978132b50 0x7f6978132f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43624/0 (socket says 172.21.15.60:43624) 2023-08-24T10:07:38.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.845+0000 7f697d647640 1 -- 172.21.15.60:0/370371625 >> v1:172.21.15.60:6789/0 conn(0x7f6978132b50 legacy=0x7f6978132f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:38.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.845+0000 7f697d647640 1 -- 172.21.15.60:0/370371625 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6978132840 con 0x7f6978135980 2023-08-24T10:07:38.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.845+0000 7f697d647640 1 --2- 172.21.15.60:0/370371625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6978135980 0x7f6978137da0 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd8bebd3e449b32 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:38.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.845+0000 7f697ce46640 1 -- 172.21.15.60:0/370371625 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f696c009dd0 con 0x7f6978135980 2023-08-24T10:07:38.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.845+0000 7f697ce46640 1 -- 172.21.15.60:0/370371625 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f696c00e230 con 0x7f6978135980 2023-08-24T10:07:38.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.845+0000 7f697ce46640 1 -- 172.21.15.60:0/370371625 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f696c00e4d0 con 0x7f6978135980 2023-08-24T10:07:38.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.846+0000 7f69800ac640 1 -- 172.21.15.60:0/370371625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6978135980 msgr2=0x7f6978137da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:38.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.846+0000 7f69800ac640 1 --2- 172.21.15.60:0/370371625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6978135980 0x7f6978137da0 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:38.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.846+0000 7f69800ac640 1 -- 172.21.15.60:0/370371625 shutdown_connections 2023-08-24T10:07:38.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.846+0000 7f69800ac640 1 --2- 172.21.15.60:0/370371625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6978135980 0x7f6978137da0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:38.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.846+0000 7f69800ac640 1 -- 172.21.15.60:0/370371625 >> 172.21.15.60:0/370371625 conn(0x7f69780a26a0 msgr2=0x7f69780a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:38.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.846+0000 7f69800ac640 1 -- 172.21.15.60:0/370371625 shutdown_connections 2023-08-24T10:07:38.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.846+0000 7f69800ac640 1 -- 172.21.15.60:0/370371625 wait complete. 2023-08-24T10:07:38.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.846+0000 7f69800ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:38.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.847+0000 7f69800ac640 1 Processor -- start 2023-08-24T10:07:38.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.847+0000 7f69800ac640 1 -- start start 2023-08-24T10:07:38.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.848+0000 7f69800ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6978132b50 0x7f69780a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:38.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.848+0000 7f69800ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69780a1b60 con 0x7f6978132b50 2023-08-24T10:07:38.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.848+0000 7f697de48640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6978132b50 0x7f69780a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:38.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.848+0000 7f697de48640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6978132b50 0x7f69780a0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50108/0 (socket says 172.21.15.60:50108) 2023-08-24T10:07:38.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.848+0000 7f697de48640 1 -- 172.21.15.60:0/2006477033 learned_addr learned my addr 172.21.15.60:0/2006477033 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:38.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.848+0000 7f697de48640 1 -- 172.21.15.60:0/2006477033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697809fab0 con 0x7f6978132b50 2023-08-24T10:07:38.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.848+0000 7f697de48640 1 --2- 172.21.15.60:0/2006477033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6978132b50 0x7f69780a0000 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:38.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.848+0000 7f69767fc640 1 -- 172.21.15.60:0/2006477033 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f69700071e0 con 0x7f6978132b50 2023-08-24T10:07:38.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.848+0000 7f69767fc640 1 -- 172.21.15.60:0/2006477033 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f6970007340 con 0x7f6978132b50 2023-08-24T10:07:38.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.848+0000 7f69800ac640 1 -- 172.21.15.60:0/2006477033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697806a240 con 0x7f6978132b50 2023-08-24T10:07:38.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.848+0000 7f69767fc640 1 -- 172.21.15.60:0/2006477033 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f69700075e0 con 0x7f6978132b50 2023-08-24T10:07:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.848+0000 7f69800ac640 1 -- 172.21.15.60:0/2006477033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697811ac50 con 0x7f6978132b50 2023-08-24T10:07:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.849+0000 7f69767fc640 1 -- 172.21.15.60:0/2006477033 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f69700057e0 con 0x7f6978132b50 2023-08-24T10:07:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.849+0000 7f69800ac640 1 -- 172.21.15.60:0/2006477033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6978061800 con 0x7f6978132b50 2023-08-24T10:07:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.850+0000 7f69767fc640 1 --2- 172.21.15.60:0/2006477033 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6964037130 0x7f6964039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.850+0000 7f69767fc640 1 -- 172.21.15.60:0/2006477033 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f6970046110 con 0x7f6978132b50 2023-08-24T10:07:38.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.850+0000 7f697d647640 1 --2- 172.21.15.60:0/2006477033 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6964037130 0x7f6964039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:38.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.850+0000 7f697d647640 1 --2- 172.21.15.60:0/2006477033 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6964037130 0x7f6964039640 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:38.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:38.854+0000 7f69767fc640 1 -- 172.21.15.60:0/2006477033 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f697000d330 con 0x7f6978132b50 2023-08-24T10:07:39.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:39.026+0000 7f69800ac640 1 -- 172.21.15.60:0/2006477033 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f697809f940 con 0x7f6964037130 2023-08-24T10:07:39.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:39.028+0000 7f69767fc640 1 -- 172.21.15.60:0/2006477033 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f697809f940 con 0x7f6964037130 2023-08-24T10:07:39.031 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:39.032 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:39.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:39.029+0000 7f69800ac640 1 -- 172.21.15.60:0/2006477033 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6964037130 msgr2=0x7f6964039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:39.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:39.029+0000 7f69800ac640 1 --2- 172.21.15.60:0/2006477033 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6964037130 0x7f6964039640 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:39.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:39.029+0000 7f69800ac640 1 -- 172.21.15.60:0/2006477033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6978132b50 msgr2=0x7f69780a0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:39.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:39.029+0000 7f69800ac640 1 --2- 172.21.15.60:0/2006477033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6978132b50 0x7f69780a0000 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:39.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:39.029+0000 7f69800ac640 1 -- 172.21.15.60:0/2006477033 shutdown_connections 2023-08-24T10:07:39.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:39.029+0000 7f69800ac640 1 --2- 172.21.15.60:0/2006477033 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6964037130 0x7f6964039640 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:39.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:39.029+0000 7f69800ac640 1 --2- 172.21.15.60:0/2006477033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6978132b50 0x7f69780a0000 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:39.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:39.029+0000 7f69800ac640 1 -- 172.21.15.60:0/2006477033 >> 172.21.15.60:0/2006477033 conn(0x7f69780a26a0 msgr2=0x7f69781baaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:39.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:39.029+0000 7f69800ac640 1 -- 172.21.15.60:0/2006477033 shutdown_connections 2023-08-24T10:07:39.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:39.029+0000 7f69800ac640 1 -- 172.21.15.60:0/2006477033 wait complete. 2023-08-24T10:07:39.041 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2023-08-24T10:07:37.526467+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:42.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.164+0000 7f0d5299e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:42.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.169+0000 7f0d5299e640 1 Processor -- start 2023-08-24T10:07:42.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.169+0000 7f0d5299e640 1 -- start start 2023-08-24T10:07:42.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.169+0000 7f0d5299e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0d4c135980 0x7f0d4c137da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:42.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.169+0000 7f0d5299e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d4c132610 con 0x7f0d4c132b50 2023-08-24T10:07:42.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.169+0000 7f0d5299e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d4c09c260 con 0x7f0d4c135980 2023-08-24T10:07:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.170+0000 7f0d4b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0d4c135980 0x7f0d4c137da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.170+0000 7f0d4b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0d4c135980 0x7f0d4c137da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50114/0 (socket says 172.21.15.60:50114) 2023-08-24T10:07:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.170+0000 7f0d4b7fe640 1 -- 172.21.15.60:0/2261414572 learned_addr learned my addr 172.21.15.60:0/2261414572 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.170+0000 7f0d4b7fe640 1 -- 172.21.15.60:0/2261414572 >> v1:172.21.15.60:6789/0 conn(0x7f0d4c132b50 legacy=0x7f0d4c132f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:42.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.170+0000 7f0d4b7fe640 1 -- 172.21.15.60:0/2261414572 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c132840 con 0x7f0d4c135980 2023-08-24T10:07:42.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.170+0000 7f0d4b7fe640 1 --2- 172.21.15.60:0/2261414572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d4c135980 0x7f0d4c137da0 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43348dc8d1bfe0fd server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:42.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.170+0000 7f0d4affd640 1 -- 172.21.15.60:0/2261414572 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0d40009dd0 con 0x7f0d4c135980 2023-08-24T10:07:42.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.170+0000 7f0d4affd640 1 -- 172.21.15.60:0/2261414572 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0d4000e230 con 0x7f0d4c135980 2023-08-24T10:07:42.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.170+0000 7f0d4affd640 1 -- 172.21.15.60:0/2261414572 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0d4000e4d0 con 0x7f0d4c135980 2023-08-24T10:07:42.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.171+0000 7f0d5299e640 1 -- 172.21.15.60:0/2261414572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d4c135980 msgr2=0x7f0d4c137da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:42.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.171+0000 7f0d5299e640 1 --2- 172.21.15.60:0/2261414572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d4c135980 0x7f0d4c137da0 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:42.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.171+0000 7f0d5299e640 1 -- 172.21.15.60:0/2261414572 shutdown_connections 2023-08-24T10:07:42.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.171+0000 7f0d5299e640 1 --2- 172.21.15.60:0/2261414572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d4c135980 0x7f0d4c137da0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:42.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.171+0000 7f0d5299e640 1 -- 172.21.15.60:0/2261414572 >> 172.21.15.60:0/2261414572 conn(0x7f0d4c0a26a0 msgr2=0x7f0d4c0a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:42.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.171+0000 7f0d5299e640 1 -- 172.21.15.60:0/2261414572 shutdown_connections 2023-08-24T10:07:42.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.171+0000 7f0d5299e640 1 -- 172.21.15.60:0/2261414572 wait complete. 2023-08-24T10:07:42.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.171+0000 7f0d5299e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:42.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.172+0000 7f0d5299e640 1 Processor -- start 2023-08-24T10:07:42.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.172+0000 7f0d5299e640 1 -- start start 2023-08-24T10:07:42.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.173+0000 7f0d5299e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d4c132b50 0x7f0d4c0a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:42.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.173+0000 7f0d5299e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c0a1b60 con 0x7f0d4c132b50 2023-08-24T10:07:42.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.173+0000 7f0d4bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d4c132b50 0x7f0d4c0a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:42.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.173+0000 7f0d4bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d4c132b50 0x7f0d4c0a0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50120/0 (socket says 172.21.15.60:50120) 2023-08-24T10:07:42.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.173+0000 7f0d4bfff640 1 -- 172.21.15.60:0/2202861239 learned_addr learned my addr 172.21.15.60:0/2202861239 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:42.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.174+0000 7f0d4bfff640 1 -- 172.21.15.60:0/2202861239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c09fab0 con 0x7f0d4c132b50 2023-08-24T10:07:42.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.174+0000 7f0d4bfff640 1 --2- 172.21.15.60:0/2202861239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d4c132b50 0x7f0d4c0a0000 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:42.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.174+0000 7f0d48ff9640 1 -- 172.21.15.60:0/2202861239 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0d3c00c860 con 0x7f0d4c132b50 2023-08-24T10:07:42.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.174+0000 7f0d48ff9640 1 -- 172.21.15.60:0/2202861239 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0d3c00c9c0 con 0x7f0d4c132b50 2023-08-24T10:07:42.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.174+0000 7f0d5299e640 1 -- 172.21.15.60:0/2202861239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4c06a240 con 0x7f0d4c132b50 2023-08-24T10:07:42.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.174+0000 7f0d48ff9640 1 -- 172.21.15.60:0/2202861239 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0d3c00cc60 con 0x7f0d4c132b50 2023-08-24T10:07:42.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.174+0000 7f0d5299e640 1 -- 172.21.15.60:0/2202861239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4c11ac50 con 0x7f0d4c132b50 2023-08-24T10:07:42.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.175+0000 7f0d48ff9640 1 -- 172.21.15.60:0/2202861239 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f0d3c011310 con 0x7f0d4c132b50 2023-08-24T10:07:42.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.175+0000 7f0d5299e640 1 -- 172.21.15.60:0/2202861239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4c061800 con 0x7f0d4c132b50 2023-08-24T10:07:42.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.175+0000 7f0d48ff9640 1 --2- 172.21.15.60:0/2202861239 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0d380370e0 0x7f0d380395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:42.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.175+0000 7f0d48ff9640 1 -- 172.21.15.60:0/2202861239 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f0d3c046960 con 0x7f0d4c132b50 2023-08-24T10:07:42.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.175+0000 7f0d4b7fe640 1 --2- 172.21.15.60:0/2202861239 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0d380370e0 0x7f0d380395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:42.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.176+0000 7f0d4b7fe640 1 --2- 172.21.15.60:0/2202861239 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0d380370e0 0x7f0d380395f0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:42.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.180+0000 7f0d48ff9640 1 -- 172.21.15.60:0/2202861239 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f0d3c00a980 con 0x7f0d4c132b50 2023-08-24T10:07:42.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.353+0000 7f0d5299e640 1 -- 172.21.15.60:0/2202861239 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d4c09f940 con 0x7f0d380370e0 2023-08-24T10:07:42.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.355+0000 7f0d48ff9640 1 -- 172.21.15.60:0/2202861239 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f0d4c09f940 con 0x7f0d380370e0 2023-08-24T10:07:42.356 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:42.357 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:42.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.357+0000 7f0d5299e640 1 -- 172.21.15.60:0/2202861239 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0d380370e0 msgr2=0x7f0d380395f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:42.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.357+0000 7f0d5299e640 1 --2- 172.21.15.60:0/2202861239 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0d380370e0 0x7f0d380395f0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:42.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.357+0000 7f0d5299e640 1 -- 172.21.15.60:0/2202861239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d4c132b50 msgr2=0x7f0d4c0a0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:42.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.357+0000 7f0d5299e640 1 --2- 172.21.15.60:0/2202861239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d4c132b50 0x7f0d4c0a0000 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:42.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.357+0000 7f0d5299e640 1 -- 172.21.15.60:0/2202861239 shutdown_connections 2023-08-24T10:07:42.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.357+0000 7f0d5299e640 1 --2- 172.21.15.60:0/2202861239 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0d380370e0 0x7f0d380395f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:42.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.357+0000 7f0d5299e640 1 --2- 172.21.15.60:0/2202861239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d4c132b50 0x7f0d4c0a0000 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:42.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.357+0000 7f0d5299e640 1 -- 172.21.15.60:0/2202861239 >> 172.21.15.60:0/2202861239 conn(0x7f0d4c0a26a0 msgr2=0x7f0d4c1baaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:42.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.357+0000 7f0d5299e640 1 -- 172.21.15.60:0/2202861239 shutdown_connections 2023-08-24T10:07:42.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.357+0000 7f0d5299e640 1 -- 172.21.15.60:0/2202861239 wait complete. 2023-08-24T10:07:42.368 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2023-08-24T10:07:41.527128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:42.370 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:42.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.478+0000 7f2631856640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:42.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.482+0000 7f2631856640 1 Processor -- start 2023-08-24T10:07:42.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.482+0000 7f2631856640 1 -- start start 2023-08-24T10:07:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.483+0000 7f2631856640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f262c096860 0x7f262c096c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.483+0000 7f2631856640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f262c0971b0 con 0x7f262c0990b0 2023-08-24T10:07:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.483+0000 7f2631856640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f262c06db70 con 0x7f262c096860 2023-08-24T10:07:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.483+0000 7f262affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f262c096860 0x7f262c096c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.483+0000 7f262affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f262c096860 0x7f262c096c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50126/0 (socket says 172.21.15.60:50126) 2023-08-24T10:07:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.483+0000 7f262affd640 1 -- 172.21.15.60:0/1489375721 learned_addr learned my addr 172.21.15.60:0/1489375721 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:42.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.483+0000 7f262affd640 1 -- 172.21.15.60:0/1489375721 >> v1:172.21.15.60:6789/0 conn(0x7f262c0990b0 legacy=0x7f262c0994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:42.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.483+0000 7f262affd640 1 -- 172.21.15.60:0/1489375721 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c06e320 con 0x7f262c096860 2023-08-24T10:07:42.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.483+0000 7f262affd640 1 --2- 172.21.15.60:0/1489375721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262c096860 0x7f262c096c70 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2720d5189c89ce04 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:42.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.483+0000 7f262a7fc640 1 -- 172.21.15.60:0/1489375721 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f261c010570 con 0x7f262c096860 2023-08-24T10:07:42.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.483+0000 7f262a7fc640 1 -- 172.21.15.60:0/1489375721 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f261c0106d0 con 0x7f262c096860 2023-08-24T10:07:42.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.483+0000 7f262a7fc640 1 -- 172.21.15.60:0/1489375721 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f261c010970 con 0x7f262c096860 2023-08-24T10:07:42.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.484+0000 7f2631856640 1 -- 172.21.15.60:0/1489375721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262c096860 msgr2=0x7f262c096c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:42.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.484+0000 7f2631856640 1 --2- 172.21.15.60:0/1489375721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262c096860 0x7f262c096c70 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:42.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.484+0000 7f2631856640 1 -- 172.21.15.60:0/1489375721 shutdown_connections 2023-08-24T10:07:42.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.484+0000 7f2631856640 1 --2- 172.21.15.60:0/1489375721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262c096860 0x7f262c096c70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:42.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.484+0000 7f2631856640 1 -- 172.21.15.60:0/1489375721 >> 172.21.15.60:0/1489375721 conn(0x7f262c09e110 msgr2=0x7f262c09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:42.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.484+0000 7f2631856640 1 -- 172.21.15.60:0/1489375721 shutdown_connections 2023-08-24T10:07:42.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.484+0000 7f2631856640 1 -- 172.21.15.60:0/1489375721 wait complete. 2023-08-24T10:07:42.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.484+0000 7f2631856640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:42.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.485+0000 7f2631856640 1 Processor -- start 2023-08-24T10:07:42.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.485+0000 7f2631856640 1 -- start start 2023-08-24T10:07:42.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.486+0000 7f2631856640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262c096860 0x7f262c12c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:42.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.486+0000 7f2631856640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c06b150 con 0x7f262c096860 2023-08-24T10:07:42.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.486+0000 7f262affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262c096860 0x7f262c12c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:42.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.486+0000 7f262affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262c096860 0x7f262c12c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50140/0 (socket says 172.21.15.60:50140) 2023-08-24T10:07:42.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.486+0000 7f262affd640 1 -- 172.21.15.60:0/3195346849 learned_addr learned my addr 172.21.15.60:0/3195346849 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:42.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.486+0000 7f262affd640 1 -- 172.21.15.60:0/3195346849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c130190 con 0x7f262c096860 2023-08-24T10:07:42.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.486+0000 7f262affd640 1 --2- 172.21.15.60:0/3195346849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262c096860 0x7f262c12c150 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:42.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.487+0000 7f2628ff9640 1 -- 172.21.15.60:0/3195346849 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f261c00f930 con 0x7f262c096860 2023-08-24T10:07:42.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.487+0000 7f2628ff9640 1 -- 172.21.15.60:0/3195346849 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f261c00fa90 con 0x7f262c096860 2023-08-24T10:07:42.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.487+0000 7f2631856640 1 -- 172.21.15.60:0/3195346849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262c12ed10 con 0x7f262c096860 2023-08-24T10:07:42.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.487+0000 7f2628ff9640 1 -- 172.21.15.60:0/3195346849 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f261c00fd30 con 0x7f262c096860 2023-08-24T10:07:42.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.487+0000 7f2631856640 1 -- 172.21.15.60:0/3195346849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262c093e20 con 0x7f262c096860 2023-08-24T10:07:42.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.487+0000 7f2628ff9640 1 -- 172.21.15.60:0/3195346849 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f261c00e360 con 0x7f262c096860 2023-08-24T10:07:42.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.488+0000 7f2628ff9640 1 --2- 172.21.15.60:0/3195346849 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f260c037130 0x7f260c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:42.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.488+0000 7f2628ff9640 1 -- 172.21.15.60:0/3195346849 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f261c016040 con 0x7f262c096860 2023-08-24T10:07:42.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.488+0000 7f2631856640 1 -- 172.21.15.60:0/3195346849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f262c12ef00 con 0x7f262c096860 2023-08-24T10:07:42.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.488+0000 7f2623fff640 1 --2- 172.21.15.60:0/3195346849 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f260c037130 0x7f260c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:42.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.488+0000 7f2623fff640 1 --2- 172.21.15.60:0/3195346849 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f260c037130 0x7f260c039640 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:42.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.492+0000 7f2628ff9640 1 -- 172.21.15.60:0/3195346849 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f262c12ef00 con 0x7f262c096860 2023-08-24T10:07:42.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.667+0000 7f2631856640 1 -- 172.21.15.60:0/3195346849 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f262c06e5a0 con 0x7f260c037130 2023-08-24T10:07:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.669+0000 7f2628ff9640 1 -- 172.21.15.60:0/3195346849 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f2614001650 con 0x7f260c037130 2023-08-24T10:07:42.670 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.670+0000 7f2631856640 1 -- 172.21.15.60:0/3195346849 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f260c037130 msgr2=0x7f260c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.671+0000 7f2631856640 1 --2- 172.21.15.60:0/3195346849 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f260c037130 0x7f260c039640 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.671+0000 7f2631856640 1 -- 172.21.15.60:0/3195346849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262c096860 msgr2=0x7f262c12c150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.671+0000 7f2631856640 1 --2- 172.21.15.60:0/3195346849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262c096860 0x7f262c12c150 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.671+0000 7f2631856640 1 -- 172.21.15.60:0/3195346849 shutdown_connections 2023-08-24T10:07:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.671+0000 7f2631856640 1 --2- 172.21.15.60:0/3195346849 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f260c037130 0x7f260c039640 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.671+0000 7f2631856640 1 --2- 172.21.15.60:0/3195346849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262c096860 0x7f262c12c150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:42.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.671+0000 7f2631856640 1 -- 172.21.15.60:0/3195346849 >> 172.21.15.60:0/3195346849 conn(0x7f262c09e110 msgr2=0x7f262c1b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:42.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.671+0000 7f2631856640 1 -- 172.21.15.60:0/3195346849 shutdown_connections 2023-08-24T10:07:42.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.671+0000 7f2631856640 1 -- 172.21.15.60:0/3195346849 wait complete. 2023-08-24T10:07:42.682 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2023-08-24T10:07:41.527128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:42.683 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:07:42.684 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:42.684 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:07:42.684 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:42.684 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:07:42.685 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:42.685 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:07:42.685 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:42.685 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:07:42.686 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:42.686 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:07:42.686 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:42.686 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:07:42.687 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:42.687 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:07:42.687 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:42.687 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:07:42.688 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:42.688 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:07:42.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.795+0000 7f0c8eb9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:42.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.797+0000 7f0c8eb9e640 1 Processor -- start 2023-08-24T10:07:42.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.798+0000 7f0c8eb9e640 1 -- start start 2023-08-24T10:07:42.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.798+0000 7f0c8eb9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0c8812ece0 0x7f0c8812f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:42.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.798+0000 7f0c8eb9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c8809a650 con 0x7f0c8812f630 2023-08-24T10:07:42.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.798+0000 7f0c8eb9e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c8806da00 con 0x7f0c8812ece0 2023-08-24T10:07:42.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.798+0000 7f0c8c93a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0c8812ece0 0x7f0c8812f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:42.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.798+0000 7f0c8c93a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0c8812ece0 0x7f0c8812f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50156/0 (socket says 172.21.15.60:50156) 2023-08-24T10:07:42.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.798+0000 7f0c8c93a640 1 -- 172.21.15.60:0/1447058880 learned_addr learned my addr 172.21.15.60:0/1447058880 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:42.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.798+0000 7f0c87fff640 1 --1- 172.21.15.60:0/1447058880 >> v1:172.21.15.60:6789/0 conn(0x7f0c8812f630 0x7f0c88097fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43652/0 (socket says 172.21.15.60:43652) 2023-08-24T10:07:42.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.799+0000 7f0c8c93a640 1 -- 172.21.15.60:0/1447058880 >> v1:172.21.15.60:6789/0 conn(0x7f0c8812f630 legacy=0x7f0c88097fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:42.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.799+0000 7f0c8c93a640 1 -- 172.21.15.60:0/1447058880 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8812ea10 con 0x7f0c8812ece0 2023-08-24T10:07:42.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.799+0000 7f0c8c93a640 1 --2- 172.21.15.60:0/1447058880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c8812ece0 0x7f0c8812f0f0 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dad2c2b71aad03b server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:42.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.799+0000 7f0c877fe640 1 -- 172.21.15.60:0/1447058880 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0c8000e040 con 0x7f0c8812ece0 2023-08-24T10:07:42.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.799+0000 7f0c877fe640 1 -- 172.21.15.60:0/1447058880 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0c800091e0 con 0x7f0c8812ece0 2023-08-24T10:07:42.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.799+0000 7f0c877fe640 1 -- 172.21.15.60:0/1447058880 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0c80007730 con 0x7f0c8812ece0 2023-08-24T10:07:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.800+0000 7f0c8eb9e640 1 -- 172.21.15.60:0/1447058880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c8812ece0 msgr2=0x7f0c8812f0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.800+0000 7f0c8eb9e640 1 --2- 172.21.15.60:0/1447058880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c8812ece0 0x7f0c8812f0f0 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.800+0000 7f0c8eb9e640 1 -- 172.21.15.60:0/1447058880 shutdown_connections 2023-08-24T10:07:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.800+0000 7f0c8eb9e640 1 --2- 172.21.15.60:0/1447058880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c8812ece0 0x7f0c8812f0f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.800+0000 7f0c8eb9e640 1 -- 172.21.15.60:0/1447058880 >> 172.21.15.60:0/1447058880 conn(0x7f0c8809e110 msgr2=0x7f0c8809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:42.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.800+0000 7f0c8eb9e640 1 -- 172.21.15.60:0/1447058880 shutdown_connections 2023-08-24T10:07:42.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.800+0000 7f0c8eb9e640 1 -- 172.21.15.60:0/1447058880 wait complete. 2023-08-24T10:07:42.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.800+0000 7f0c8eb9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:42.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.801+0000 7f0c8eb9e640 1 Processor -- start 2023-08-24T10:07:42.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.801+0000 7f0c8eb9e640 1 -- start start 2023-08-24T10:07:42.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.802+0000 7f0c8eb9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c8812ece0 0x7f0c881330e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:42.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.802+0000 7f0c8eb9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c88125c50 con 0x7f0c8812ece0 2023-08-24T10:07:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.802+0000 7f0c8c93a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c8812ece0 0x7f0c881330e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.802+0000 7f0c8c93a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c8812ece0 0x7f0c881330e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50166/0 (socket says 172.21.15.60:50166) 2023-08-24T10:07:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.802+0000 7f0c8c93a640 1 -- 172.21.15.60:0/2031359289 learned_addr learned my addr 172.21.15.60:0/2031359289 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.802+0000 7f0c8c93a640 1 -- 172.21.15.60:0/2031359289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c88126c10 con 0x7f0c8812ece0 2023-08-24T10:07:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.802+0000 7f0c8c93a640 1 --2- 172.21.15.60:0/2031359289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c8812ece0 0x7f0c881330e0 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:42.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.802+0000 7f0c857fa640 1 -- 172.21.15.60:0/2031359289 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0c8000e040 con 0x7f0c8812ece0 2023-08-24T10:07:42.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.802+0000 7f0c857fa640 1 -- 172.21.15.60:0/2031359289 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0c80010830 con 0x7f0c8812ece0 2023-08-24T10:07:42.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.802+0000 7f0c8eb9e640 1 -- 172.21.15.60:0/2031359289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c881252d0 con 0x7f0c8812ece0 2023-08-24T10:07:42.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.802+0000 7f0c857fa640 1 -- 172.21.15.60:0/2031359289 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0c80010ad0 con 0x7f0c8812ece0 2023-08-24T10:07:42.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.802+0000 7f0c8eb9e640 1 -- 172.21.15.60:0/2031359289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c881272c0 con 0x7f0c8812ece0 2023-08-24T10:07:42.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.803+0000 7f0c857fa640 1 -- 172.21.15.60:0/2031359289 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f0c80010c30 con 0x7f0c8812ece0 2023-08-24T10:07:42.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.803+0000 7f0c857fa640 1 --2- 172.21.15.60:0/2031359289 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0c74037130 0x7f0c74039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:42.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.803+0000 7f0c857fa640 1 -- 172.21.15.60:0/2031359289 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f0c800479d0 con 0x7f0c8812ece0 2023-08-24T10:07:42.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.803+0000 7f0c8eb9e640 1 -- 172.21.15.60:0/2031359289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c88061800 con 0x7f0c8812ece0 2023-08-24T10:07:42.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.804+0000 7f0c87fff640 1 --2- 172.21.15.60:0/2031359289 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0c74037130 0x7f0c74039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:42.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.804+0000 7f0c87fff640 1 --2- 172.21.15.60:0/2031359289 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0c74037130 0x7f0c74039640 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:42.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:42.808+0000 7f0c857fa640 1 -- 172.21.15.60:0/2031359289 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f0c8000e5b0 con 0x7f0c8812ece0 2023-08-24T10:07:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.054+0000 7f0c8eb9e640 1 -- 172.21.15.60:0/2031359289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0c88073730 con 0x7f0c8812ece0 2023-08-24T10:07:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.054+0000 7f0c857fa640 1 -- 172.21.15.60:0/2031359289 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f0c8000e1a0 con 0x7f0c8812ece0 2023-08-24T10:07:43.056 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:43.056 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":173,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:07:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.056+0000 7f0c8eb9e640 1 -- 172.21.15.60:0/2031359289 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0c74037130 msgr2=0x7f0c74039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.056+0000 7f0c8eb9e640 1 --2- 172.21.15.60:0/2031359289 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0c74037130 0x7f0c74039640 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.056+0000 7f0c8eb9e640 1 -- 172.21.15.60:0/2031359289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c8812ece0 msgr2=0x7f0c881330e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.056+0000 7f0c8eb9e640 1 --2- 172.21.15.60:0/2031359289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c8812ece0 0x7f0c881330e0 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.056+0000 7f0c8eb9e640 1 -- 172.21.15.60:0/2031359289 shutdown_connections 2023-08-24T10:07:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.056+0000 7f0c8eb9e640 1 --2- 172.21.15.60:0/2031359289 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0c74037130 0x7f0c74039640 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.056+0000 7f0c8eb9e640 1 --2- 172.21.15.60:0/2031359289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c8812ece0 0x7f0c881330e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.056+0000 7f0c8eb9e640 1 -- 172.21.15.60:0/2031359289 >> 172.21.15.60:0/2031359289 conn(0x7f0c8809e110 msgr2=0x7f0c88097710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:43.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.056+0000 7f0c8eb9e640 1 -- 172.21.15.60:0/2031359289 shutdown_connections 2023-08-24T10:07:43.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.056+0000 7f0c8eb9e640 1 -- 172.21.15.60:0/2031359289 wait complete. 2023-08-24T10:07:43.068 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:07:43.068 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:43.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.174+0000 7fc11bb9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:43.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.178+0000 7fc11bb9e640 1 Processor -- start 2023-08-24T10:07:43.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.178+0000 7fc11bb9e640 1 -- start start 2023-08-24T10:07:43.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.179+0000 7fc11bb9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc11412d360 0x7fc11412d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:43.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.179+0000 7fc11bb9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11412cfa0 con 0x7fc1141304a0 2023-08-24T10:07:43.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.179+0000 7fc11bb9e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc114070d70 con 0x7fc11412d360 2023-08-24T10:07:43.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.179+0000 7fc11993a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc11412d360 0x7fc11412d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:43.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.179+0000 7fc11993a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc11412d360 0x7fc11412d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50180/0 (socket says 172.21.15.60:50180) 2023-08-24T10:07:43.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.179+0000 7fc11993a640 1 -- 172.21.15.60:0/1422275586 learned_addr learned my addr 172.21.15.60:0/1422275586 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:43.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.179+0000 7fc119139640 1 --1- 172.21.15.60:0/1422275586 >> v1:172.21.15.60:6789/0 conn(0x7fc1141304a0 0x7fc1141308b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43656/0 (socket says 172.21.15.60:43656) 2023-08-24T10:07:43.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.180+0000 7fc11993a640 1 -- 172.21.15.60:0/1422275586 >> v1:172.21.15.60:6789/0 conn(0x7fc1141304a0 legacy=0x7fc1141308b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:43.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.180+0000 7fc11993a640 1 -- 172.21.15.60:0/1422275586 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11409a5d0 con 0x7fc11412d360 2023-08-24T10:07:43.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.180+0000 7fc11993a640 1 --2- 172.21.15.60:0/1422275586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11412d360 0x7fc11412d770 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61dacb653f5526d0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:43.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.180+0000 7fc118938640 1 -- 172.21.15.60:0/1422275586 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc110009dd0 con 0x7fc11412d360 2023-08-24T10:07:43.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.180+0000 7fc118938640 1 -- 172.21.15.60:0/1422275586 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc11000e230 con 0x7fc11412d360 2023-08-24T10:07:43.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.180+0000 7fc118938640 1 -- 172.21.15.60:0/1422275586 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc11000e4d0 con 0x7fc11412d360 2023-08-24T10:07:43.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.180+0000 7fc11bb9e640 1 -- 172.21.15.60:0/1422275586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11412d360 msgr2=0x7fc11412d770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:43.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.180+0000 7fc11bb9e640 1 --2- 172.21.15.60:0/1422275586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11412d360 0x7fc11412d770 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:43.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.181+0000 7fc11bb9e640 1 -- 172.21.15.60:0/1422275586 shutdown_connections 2023-08-24T10:07:43.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.181+0000 7fc11bb9e640 1 --2- 172.21.15.60:0/1422275586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11412d360 0x7fc11412d770 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:43.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.181+0000 7fc11bb9e640 1 -- 172.21.15.60:0/1422275586 >> 172.21.15.60:0/1422275586 conn(0x7fc11409e610 msgr2=0x7fc11409ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:43.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.181+0000 7fc11bb9e640 1 -- 172.21.15.60:0/1422275586 shutdown_connections 2023-08-24T10:07:43.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.181+0000 7fc11bb9e640 1 -- 172.21.15.60:0/1422275586 wait complete. 2023-08-24T10:07:43.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.181+0000 7fc11bb9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:43.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.182+0000 7fc11bb9e640 1 Processor -- start 2023-08-24T10:07:43.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.182+0000 7fc11bb9e640 1 -- start start 2023-08-24T10:07:43.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.182+0000 7fc11bb9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11412d360 0x7fc114118ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:43.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.182+0000 7fc11bb9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc114118970 con 0x7fc11412d360 2023-08-24T10:07:43.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.183+0000 7fc11993a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11412d360 0x7fc114118ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:43.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.183+0000 7fc11993a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11412d360 0x7fc114118ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50188/0 (socket says 172.21.15.60:50188) 2023-08-24T10:07:43.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.183+0000 7fc11993a640 1 -- 172.21.15.60:0/2141232254 learned_addr learned my addr 172.21.15.60:0/2141232254 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:43.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.183+0000 7fc11993a640 1 -- 172.21.15.60:0/2141232254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc114118660 con 0x7fc11412d360 2023-08-24T10:07:43.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.183+0000 7fc11993a640 1 --2- 172.21.15.60:0/2141232254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11412d360 0x7fc114118ea0 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:43.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.183+0000 7fc10a7fc640 1 -- 172.21.15.60:0/2141232254 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc110014410 con 0x7fc11412d360 2023-08-24T10:07:43.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.183+0000 7fc10a7fc640 1 -- 172.21.15.60:0/2141232254 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc110014570 con 0x7fc11412d360 2023-08-24T10:07:43.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.183+0000 7fc11bb9e640 1 -- 172.21.15.60:0/2141232254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11411c630 con 0x7fc11412d360 2023-08-24T10:07:43.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.183+0000 7fc10a7fc640 1 -- 172.21.15.60:0/2141232254 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc110014810 con 0x7fc11412d360 2023-08-24T10:07:43.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.183+0000 7fc11bb9e640 1 -- 172.21.15.60:0/2141232254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11411bbc0 con 0x7fc11412d360 2023-08-24T10:07:43.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.184+0000 7fc10a7fc640 1 -- 172.21.15.60:0/2141232254 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fc110014a10 con 0x7fc11412d360 2023-08-24T10:07:43.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.184+0000 7fc11bb9e640 1 -- 172.21.15.60:0/2141232254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc114061800 con 0x7fc11412d360 2023-08-24T10:07:43.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.184+0000 7fc10a7fc640 1 --2- 172.21.15.60:0/2141232254 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc0f80370e0 0x7fc0f80395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:43.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.184+0000 7fc10a7fc640 1 -- 172.21.15.60:0/2141232254 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fc110019040 con 0x7fc11412d360 2023-08-24T10:07:43.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.185+0000 7fc119139640 1 --2- 172.21.15.60:0/2141232254 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc0f80370e0 0x7fc0f80395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:43.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.185+0000 7fc119139640 1 --2- 172.21.15.60:0/2141232254 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc0f80370e0 0x7fc0f80395f0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:43.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.189+0000 7fc10a7fc640 1 -- 172.21.15.60:0/2141232254 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fc110018190 con 0x7fc11412d360 2023-08-24T10:07:43.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.363+0000 7fc11bb9e640 1 -- 172.21.15.60:0/2141232254 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc114070630 con 0x7fc0f80370e0 2023-08-24T10:07:43.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.365+0000 7fc10a7fc640 1 -- 172.21.15.60:0/2141232254 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7fc1040070b0 con 0x7fc0f80370e0 2023-08-24T10:07:43.366 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:43.367 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:43.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.366+0000 7fc11bb9e640 1 -- 172.21.15.60:0/2141232254 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc0f80370e0 msgr2=0x7fc0f80395f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:43.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.366+0000 7fc11bb9e640 1 --2- 172.21.15.60:0/2141232254 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc0f80370e0 0x7fc0f80395f0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:43.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.366+0000 7fc11bb9e640 1 -- 172.21.15.60:0/2141232254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11412d360 msgr2=0x7fc114118ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:43.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.366+0000 7fc11bb9e640 1 --2- 172.21.15.60:0/2141232254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11412d360 0x7fc114118ea0 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:43.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.366+0000 7fc11bb9e640 1 -- 172.21.15.60:0/2141232254 shutdown_connections 2023-08-24T10:07:43.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.366+0000 7fc11bb9e640 1 --2- 172.21.15.60:0/2141232254 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc0f80370e0 0x7fc0f80395f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:43.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.366+0000 7fc11bb9e640 1 --2- 172.21.15.60:0/2141232254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11412d360 0x7fc114118ea0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:43.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.366+0000 7fc11bb9e640 1 -- 172.21.15.60:0/2141232254 >> 172.21.15.60:0/2141232254 conn(0x7fc11409e610 msgr2=0x7fc1141b8540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:43.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.366+0000 7fc11bb9e640 1 -- 172.21.15.60:0/2141232254 shutdown_connections 2023-08-24T10:07:43.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:43.366+0000 7fc11bb9e640 1 -- 172.21.15.60:0/2141232254 wait complete. 2023-08-24T10:07:43.380 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2023-08-24T10:07:41.527128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:43.567 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~156s 2023-08-24T10:07:43.567 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~156s 2023-08-24T10:07:43.567 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~156s 2023-08-24T10:07:43.567 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~156s 2023-08-24T10:07:46.382 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:46.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.501+0000 7fdb6475e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:46.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.505+0000 7fdb6475e640 1 Processor -- start 2023-08-24T10:07:46.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.505+0000 7fdb6475e640 1 -- start start 2023-08-24T10:07:46.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.506+0000 7fdb6475e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdb5c132630 0x7fdb5c134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:46.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.506+0000 7fdb6475e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5c06e320 con 0x7fdb5c096860 2023-08-24T10:07:46.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.506+0000 7fdb6475e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb5c06db70 con 0x7fdb5c132630 2023-08-24T10:07:46.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.506+0000 7fdb61cf9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdb5c132630 0x7fdb5c134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:46.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.506+0000 7fdb61cf9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdb5c132630 0x7fdb5c134a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50190/0 (socket says 172.21.15.60:50190) 2023-08-24T10:07:46.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.506+0000 7fdb61cf9640 1 -- 172.21.15.60:0/3409914283 learned_addr learned my addr 172.21.15.60:0/3409914283 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:46.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.506+0000 7fdb624fa640 1 --1- 172.21.15.60:0/3409914283 >> v1:172.21.15.60:6789/0 conn(0x7fdb5c096860 0x7fdb5c096c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43662/0 (socket says 172.21.15.60:43662) 2023-08-24T10:07:46.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.507+0000 7fdb61cf9640 1 -- 172.21.15.60:0/3409914283 >> v1:172.21.15.60:6789/0 conn(0x7fdb5c096860 legacy=0x7fdb5c096c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:46.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.507+0000 7fdb61cf9640 1 -- 172.21.15.60:0/3409914283 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5c06cef0 con 0x7fdb5c132630 2023-08-24T10:07:46.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.507+0000 7fdb61cf9640 1 --2- 172.21.15.60:0/3409914283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb5c132630 0x7fdb5c134a50 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb7ad36c8bbeb25 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:46.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.507+0000 7fdb614f8640 1 -- 172.21.15.60:0/3409914283 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdb4c009dd0 con 0x7fdb5c132630 2023-08-24T10:07:46.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.507+0000 7fdb614f8640 1 -- 172.21.15.60:0/3409914283 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fdb4c00e230 con 0x7fdb5c132630 2023-08-24T10:07:46.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.507+0000 7fdb614f8640 1 -- 172.21.15.60:0/3409914283 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdb4c00e4d0 con 0x7fdb5c132630 2023-08-24T10:07:46.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.508+0000 7fdb6475e640 1 -- 172.21.15.60:0/3409914283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb5c132630 msgr2=0x7fdb5c134a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:46.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.508+0000 7fdb6475e640 1 --2- 172.21.15.60:0/3409914283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb5c132630 0x7fdb5c134a50 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:46.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.508+0000 7fdb6475e640 1 -- 172.21.15.60:0/3409914283 shutdown_connections 2023-08-24T10:07:46.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.508+0000 7fdb6475e640 1 --2- 172.21.15.60:0/3409914283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb5c132630 0x7fdb5c134a50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:46.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.508+0000 7fdb6475e640 1 -- 172.21.15.60:0/3409914283 >> 172.21.15.60:0/3409914283 conn(0x7fdb5c09e110 msgr2=0x7fdb5c09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:46.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.508+0000 7fdb6475e640 1 -- 172.21.15.60:0/3409914283 shutdown_connections 2023-08-24T10:07:46.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.508+0000 7fdb6475e640 1 -- 172.21.15.60:0/3409914283 wait complete. 2023-08-24T10:07:46.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.508+0000 7fdb6475e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:46.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.509+0000 7fdb6475e640 1 Processor -- start 2023-08-24T10:07:46.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.509+0000 7fdb6475e640 1 -- start start 2023-08-24T10:07:46.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.510+0000 7fdb6475e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb5c096860 0x7fdb5c12d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:46.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.510+0000 7fdb6475e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c12fa30 con 0x7fdb5c096860 2023-08-24T10:07:46.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.510+0000 7fdb624fa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb5c096860 0x7fdb5c12d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:46.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.510+0000 7fdb624fa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb5c096860 0x7fdb5c12d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50206/0 (socket says 172.21.15.60:50206) 2023-08-24T10:07:46.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.510+0000 7fdb624fa640 1 -- 172.21.15.60:0/2281401566 learned_addr learned my addr 172.21.15.60:0/2281401566 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:46.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.510+0000 7fdb624fa640 1 -- 172.21.15.60:0/2281401566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5c0962c0 con 0x7fdb5c096860 2023-08-24T10:07:46.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.510+0000 7fdb624fa640 1 --2- 172.21.15.60:0/2281401566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb5c096860 0x7fdb5c12d560 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:46.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.510+0000 7fdb52ffd640 1 -- 172.21.15.60:0/2281401566 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdb580071e0 con 0x7fdb5c096860 2023-08-24T10:07:46.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.510+0000 7fdb52ffd640 1 -- 172.21.15.60:0/2281401566 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fdb58007340 con 0x7fdb5c096860 2023-08-24T10:07:46.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.510+0000 7fdb6475e640 1 -- 172.21.15.60:0/2281401566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5c095f90 con 0x7fdb5c096860 2023-08-24T10:07:46.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.510+0000 7fdb52ffd640 1 -- 172.21.15.60:0/2281401566 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdb580075e0 con 0x7fdb5c096860 2023-08-24T10:07:46.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.510+0000 7fdb6475e640 1 -- 172.21.15.60:0/2281401566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5c12efe0 con 0x7fdb5c096860 2023-08-24T10:07:46.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.511+0000 7fdb52ffd640 1 -- 172.21.15.60:0/2281401566 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fdb580057e0 con 0x7fdb5c096860 2023-08-24T10:07:46.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.511+0000 7fdb6475e640 1 -- 172.21.15.60:0/2281401566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5c096120 con 0x7fdb5c096860 2023-08-24T10:07:46.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.511+0000 7fdb52ffd640 1 --2- 172.21.15.60:0/2281401566 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fdb400370e0 0x7fdb400395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:46.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.512+0000 7fdb52ffd640 1 -- 172.21.15.60:0/2281401566 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fdb58046360 con 0x7fdb5c096860 2023-08-24T10:07:46.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.512+0000 7fdb61cf9640 1 --2- 172.21.15.60:0/2281401566 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fdb400370e0 0x7fdb400395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:46.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.512+0000 7fdb61cf9640 1 --2- 172.21.15.60:0/2281401566 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fdb400370e0 0x7fdb400395f0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:46.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.516+0000 7fdb52ffd640 1 -- 172.21.15.60:0/2281401566 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fdb5c096120 con 0x7fdb5c096860 2023-08-24T10:07:46.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.688+0000 7fdb6475e640 1 -- 172.21.15.60:0/2281401566 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdb5c08a240 con 0x7fdb400370e0 2023-08-24T10:07:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.689+0000 7fdb52ffd640 1 -- 172.21.15.60:0/2281401566 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7fdb5c08a240 con 0x7fdb400370e0 2023-08-24T10:07:46.691 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:46.692 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:46.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.691+0000 7fdb6475e640 1 -- 172.21.15.60:0/2281401566 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fdb400370e0 msgr2=0x7fdb400395f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:46.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.691+0000 7fdb6475e640 1 --2- 172.21.15.60:0/2281401566 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fdb400370e0 0x7fdb400395f0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:46.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.691+0000 7fdb6475e640 1 -- 172.21.15.60:0/2281401566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb5c096860 msgr2=0x7fdb5c12d560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:46.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.691+0000 7fdb6475e640 1 --2- 172.21.15.60:0/2281401566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb5c096860 0x7fdb5c12d560 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.691+0000 7fdb6475e640 1 -- 172.21.15.60:0/2281401566 shutdown_connections 2023-08-24T10:07:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.691+0000 7fdb6475e640 1 --2- 172.21.15.60:0/2281401566 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fdb400370e0 0x7fdb400395f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.691+0000 7fdb6475e640 1 --2- 172.21.15.60:0/2281401566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb5c096860 0x7fdb5c12d560 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.691+0000 7fdb6475e640 1 -- 172.21.15.60:0/2281401566 >> 172.21.15.60:0/2281401566 conn(0x7fdb5c09e110 msgr2=0x7fdb5c06e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.691+0000 7fdb6475e640 1 -- 172.21.15.60:0/2281401566 shutdown_connections 2023-08-24T10:07:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.691+0000 7fdb6475e640 1 -- 172.21.15.60:0/2281401566 wait complete. 2023-08-24T10:07:46.703 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2023-08-24T10:07:45.527722+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:46.704 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:46.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.810+0000 7f62f435e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.814+0000 7f62f435e640 1 Processor -- start 2023-08-24T10:07:46.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.814+0000 7f62f435e640 1 -- start start 2023-08-24T10:07:46.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.815+0000 7f62f435e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f62ec071e40 0x7f62ec096800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:46.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.815+0000 7f62f435e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ec0977f0 con 0x7f62ec06f100 2023-08-24T10:07:46.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.815+0000 7f62f435e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ec09a280 con 0x7f62ec071e40 2023-08-24T10:07:46.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.815+0000 7f62f20fa640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f62ec071e40 0x7f62ec096800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:46.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.815+0000 7f62f20fa640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f62ec071e40 0x7f62ec096800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35494/0 (socket says 172.21.15.60:35494) 2023-08-24T10:07:46.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.815+0000 7f62f20fa640 1 -- 172.21.15.60:0/2587858278 learned_addr learned my addr 172.21.15.60:0/2587858278 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:46.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.815+0000 7f62f20fa640 1 -- 172.21.15.60:0/2587858278 >> v1:172.21.15.60:6789/0 conn(0x7f62ec06f100 legacy=0x7f62ec06f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:46.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.815+0000 7f62f20fa640 1 -- 172.21.15.60:0/2587858278 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec09a6b0 con 0x7f62ec071e40 2023-08-24T10:07:46.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.815+0000 7f62f20fa640 1 --2- 172.21.15.60:0/2587858278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec071e40 0x7f62ec096800 crc :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ebe05ad8013491 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:46.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.816+0000 7f62f10f8640 1 -- 172.21.15.60:0/2587858278 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f62e4009dd0 con 0x7f62ec071e40 2023-08-24T10:07:46.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.816+0000 7f62f10f8640 1 -- 172.21.15.60:0/2587858278 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f62e400e230 con 0x7f62ec071e40 2023-08-24T10:07:46.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.816+0000 7f62f10f8640 1 -- 172.21.15.60:0/2587858278 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f62e400e4d0 con 0x7f62ec071e40 2023-08-24T10:07:46.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.816+0000 7f62f435e640 1 -- 172.21.15.60:0/2587858278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec071e40 msgr2=0x7f62ec096800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:46.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.816+0000 7f62f435e640 1 --2- 172.21.15.60:0/2587858278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec071e40 0x7f62ec096800 crc :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:46.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.816+0000 7f62f435e640 1 -- 172.21.15.60:0/2587858278 shutdown_connections 2023-08-24T10:07:46.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.816+0000 7f62f435e640 1 --2- 172.21.15.60:0/2587858278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec071e40 0x7f62ec096800 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:46.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.816+0000 7f62f435e640 1 -- 172.21.15.60:0/2587858278 >> 172.21.15.60:0/2587858278 conn(0x7f62ec09e110 msgr2=0x7f62ec09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:46.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.816+0000 7f62f435e640 1 -- 172.21.15.60:0/2587858278 shutdown_connections 2023-08-24T10:07:46.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.816+0000 7f62f435e640 1 -- 172.21.15.60:0/2587858278 wait complete. 2023-08-24T10:07:46.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.817+0000 7f62f435e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:46.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.817+0000 7f62f435e640 1 Processor -- start 2023-08-24T10:07:46.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.818+0000 7f62f435e640 1 -- start start 2023-08-24T10:07:46.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.818+0000 7f62f435e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec06f100 0x7f62ec06cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:46.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.818+0000 7f62f435e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0930d0 con 0x7f62ec06f100 2023-08-24T10:07:46.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.818+0000 7f62f20fa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec06f100 0x7f62ec06cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:46.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.818+0000 7f62f20fa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec06f100 0x7f62ec06cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35498/0 (socket says 172.21.15.60:35498) 2023-08-24T10:07:46.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.818+0000 7f62f20fa640 1 -- 172.21.15.60:0/2979033662 learned_addr learned my addr 172.21.15.60:0/2979033662 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:46.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.819+0000 7f62f20fa640 1 -- 172.21.15.60:0/2979033662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec125f70 con 0x7f62ec06f100 2023-08-24T10:07:46.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.819+0000 7f62f20fa640 1 --2- 172.21.15.60:0/2979033662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec06f100 0x7f62ec06cef0 crc :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:46.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.819+0000 7f62eaffd640 1 -- 172.21.15.60:0/2979033662 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f62e4014540 con 0x7f62ec06f100 2023-08-24T10:07:46.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.819+0000 7f62eaffd640 1 -- 172.21.15.60:0/2979033662 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f62e40146a0 con 0x7f62ec06f100 2023-08-24T10:07:46.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.819+0000 7f62f435e640 1 -- 172.21.15.60:0/2979033662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ec125340 con 0x7f62ec06f100 2023-08-24T10:07:46.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.819+0000 7f62eaffd640 1 -- 172.21.15.60:0/2979033662 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f62e4014940 con 0x7f62ec06f100 2023-08-24T10:07:46.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.819+0000 7f62f435e640 1 -- 172.21.15.60:0/2979033662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec131040 con 0x7f62ec06f100 2023-08-24T10:07:46.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.820+0000 7f62f435e640 1 -- 172.21.15.60:0/2979033662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ec125530 con 0x7f62ec06f100 2023-08-24T10:07:46.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.820+0000 7f62eaffd640 1 -- 172.21.15.60:0/2979033662 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f62e4014af0 con 0x7f62ec06f100 2023-08-24T10:07:46.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.820+0000 7f62eaffd640 1 --2- 172.21.15.60:0/2979033662 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f62d8037130 0x7f62d8039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:46.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.820+0000 7f62eaffd640 1 -- 172.21.15.60:0/2979033662 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f62e4019040 con 0x7f62ec06f100 2023-08-24T10:07:46.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.821+0000 7f62f18f9640 1 --2- 172.21.15.60:0/2979033662 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f62d8037130 0x7f62d8039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:46.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.821+0000 7f62f18f9640 1 --2- 172.21.15.60:0/2979033662 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f62d8037130 0x7f62d8039640 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:46.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:46.825+0000 7f62eaffd640 1 -- 172.21.15.60:0/2979033662 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f62e4018190 con 0x7f62ec06f100 2023-08-24T10:07:47.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.000+0000 7f62f435e640 1 -- 172.21.15.60:0/2979033662 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f62ec131290 con 0x7f62d8037130 2023-08-24T10:07:47.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.002+0000 7f62eaffd640 1 -- 172.21.15.60:0/2979033662 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f62ec131290 con 0x7f62d8037130 2023-08-24T10:07:47.003 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:47.005 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.003+0000 7f62f435e640 1 -- 172.21.15.60:0/2979033662 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f62d8037130 msgr2=0x7f62d8039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.003+0000 7f62f435e640 1 --2- 172.21.15.60:0/2979033662 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f62d8037130 0x7f62d8039640 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.003+0000 7f62f435e640 1 -- 172.21.15.60:0/2979033662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec06f100 msgr2=0x7f62ec06cef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.003+0000 7f62f435e640 1 --2- 172.21.15.60:0/2979033662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec06f100 0x7f62ec06cef0 crc :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.003+0000 7f62f435e640 1 -- 172.21.15.60:0/2979033662 shutdown_connections 2023-08-24T10:07:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.003+0000 7f62f435e640 1 --2- 172.21.15.60:0/2979033662 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f62d8037130 0x7f62d8039640 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:47.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.003+0000 7f62f435e640 1 --2- 172.21.15.60:0/2979033662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62ec06f100 0x7f62ec06cef0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:47.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.003+0000 7f62f435e640 1 -- 172.21.15.60:0/2979033662 >> 172.21.15.60:0/2979033662 conn(0x7f62ec09e110 msgr2=0x7f62ec0985b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:47.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.003+0000 7f62f435e640 1 -- 172.21.15.60:0/2979033662 shutdown_connections 2023-08-24T10:07:47.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.003+0000 7f62f435e640 1 -- 172.21.15.60:0/2979033662 wait complete. 2023-08-24T10:07:47.016 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2023-08-24T10:07:45.527722+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:47.017 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:07:47.017 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:47.017 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:07:47.018 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:47.018 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:07:47.018 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:47.018 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:07:47.018 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:47.019 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:07:47.019 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:47.019 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:07:47.019 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:47.020 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:07:47.020 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:47.020 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:07:47.020 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:47.021 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:07:47.021 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:47.021 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:07:47.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.127+0000 7f5a5e26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:47.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.130+0000 7f5a5e26c640 1 Processor -- start 2023-08-24T10:07:47.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.130+0000 7f5a5e26c640 1 -- start start 2023-08-24T10:07:47.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.131+0000 7f5a5e26c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5a5812f610 0x7f5a5812fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:47.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.131+0000 7f5a5e26c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a5809d750 con 0x7f5a581326f0 2023-08-24T10:07:47.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.131+0000 7f5a5e26c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a5809d980 con 0x7f5a5812f610 2023-08-24T10:07:47.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.131+0000 7f5a57fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5a5812f610 0x7f5a5812fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.131+0000 7f5a57fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5a5812f610 0x7f5a5812fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35502/0 (socket says 172.21.15.60:35502) 2023-08-24T10:07:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.131+0000 7f5a577fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5a581326f0 0x7f5a58132b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54200/0 (socket says 172.21.15.60:54200) 2023-08-24T10:07:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.131+0000 7f5a57fff640 1 -- 172.21.15.60:0/1096526299 learned_addr learned my addr 172.21.15.60:0/1096526299 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.132+0000 7f5a57fff640 1 -- 172.21.15.60:0/1096526299 >> v1:172.21.15.60:6789/0 conn(0x7f5a581326f0 legacy=0x7f5a58132b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.132+0000 7f5a57fff640 1 -- 172.21.15.60:0/1096526299 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5809bde0 con 0x7f5a5812f610 2023-08-24T10:07:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.132+0000 7f5a57fff640 1 --2- 172.21.15.60:0/1096526299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a5812f610 0x7f5a5812fa20 crc :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894bba51703de86f server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.132+0000 7f5a56ffd640 1 -- 172.21.15.60:0/1096526299 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5a48009dc0 con 0x7f5a5812f610 2023-08-24T10:07:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.132+0000 7f5a56ffd640 1 -- 172.21.15.60:0/1096526299 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5a4800e230 con 0x7f5a5812f610 2023-08-24T10:07:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.132+0000 7f5a56ffd640 1 -- 172.21.15.60:0/1096526299 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5a4800e4d0 con 0x7f5a5812f610 2023-08-24T10:07:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.133+0000 7f5a5e26c640 1 -- 172.21.15.60:0/1096526299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a5812f610 msgr2=0x7f5a5812fa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.133+0000 7f5a5e26c640 1 --2- 172.21.15.60:0/1096526299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a5812f610 0x7f5a5812fa20 crc :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.133+0000 7f5a5e26c640 1 -- 172.21.15.60:0/1096526299 shutdown_connections 2023-08-24T10:07:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.133+0000 7f5a5e26c640 1 --2- 172.21.15.60:0/1096526299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a5812f610 0x7f5a5812fa20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.133+0000 7f5a5e26c640 1 -- 172.21.15.60:0/1096526299 >> 172.21.15.60:0/1096526299 conn(0x7f5a580a2500 msgr2=0x7f5a580a2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.133+0000 7f5a5e26c640 1 -- 172.21.15.60:0/1096526299 shutdown_connections 2023-08-24T10:07:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.133+0000 7f5a5e26c640 1 -- 172.21.15.60:0/1096526299 wait complete. 2023-08-24T10:07:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.133+0000 7f5a5e26c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.134+0000 7f5a5e26c640 1 Processor -- start 2023-08-24T10:07:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.134+0000 7f5a5e26c640 1 -- start start 2023-08-24T10:07:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.135+0000 7f5a5e26c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a5812f610 0x7f5a5809fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.135+0000 7f5a5e26c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5812cc60 con 0x7f5a5812f610 2023-08-24T10:07:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.135+0000 7f5a57fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a5812f610 0x7f5a5809fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.135+0000 7f5a57fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a5812f610 0x7f5a5809fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35518/0 (socket says 172.21.15.60:35518) 2023-08-24T10:07:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.135+0000 7f5a57fff640 1 -- 172.21.15.60:0/4166143400 learned_addr learned my addr 172.21.15.60:0/4166143400 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.135+0000 7f5a57fff640 1 -- 172.21.15.60:0/4166143400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5806a1f0 con 0x7f5a5812f610 2023-08-24T10:07:47.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.135+0000 7f5a57fff640 1 --2- 172.21.15.60:0/4166143400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a5812f610 0x7f5a5809fca0 crc :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:47.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.135+0000 7f5a54ff9640 1 -- 172.21.15.60:0/4166143400 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5a48014540 con 0x7f5a5812f610 2023-08-24T10:07:47.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.135+0000 7f5a54ff9640 1 -- 172.21.15.60:0/4166143400 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5a480146a0 con 0x7f5a5812f610 2023-08-24T10:07:47.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.135+0000 7f5a5e26c640 1 -- 172.21.15.60:0/4166143400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a5809f7e0 con 0x7f5a5812f610 2023-08-24T10:07:47.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.135+0000 7f5a54ff9640 1 -- 172.21.15.60:0/4166143400 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5a48014940 con 0x7f5a5812f610 2023-08-24T10:07:47.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.135+0000 7f5a5e26c640 1 -- 172.21.15.60:0/4166143400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a5809f4d0 con 0x7f5a5812f610 2023-08-24T10:07:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.136+0000 7f5a54ff9640 1 -- 172.21.15.60:0/4166143400 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f5a48014af0 con 0x7f5a5812f610 2023-08-24T10:07:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.136+0000 7f5a54ff9640 1 --2- 172.21.15.60:0/4166143400 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5a44037130 0x7f5a44039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.136+0000 7f5a54ff9640 1 -- 172.21.15.60:0/4166143400 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f5a48019040 con 0x7f5a5812f610 2023-08-24T10:07:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.136+0000 7f5a5e26c640 1 -- 172.21.15.60:0/4166143400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5806a1f0 con 0x7f5a5812f610 2023-08-24T10:07:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.137+0000 7f5a577fe640 1 --2- 172.21.15.60:0/4166143400 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5a44037130 0x7f5a44039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.137+0000 7f5a577fe640 1 --2- 172.21.15.60:0/4166143400 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5a44037130 0x7f5a44039640 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.141+0000 7f5a54ff9640 1 -- 172.21.15.60:0/4166143400 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f5a5806a1f0 con 0x7f5a5812f610 2023-08-24T10:07:47.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.388+0000 7f5a5e26c640 1 -- 172.21.15.60:0/4166143400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f5a5806a1f0 con 0x7f5a5812f610 2023-08-24T10:07:47.390 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.388+0000 7f5a54ff9640 1 -- 172.21.15.60:0/4166143400 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f5a5806a1f0 con 0x7f5a5812f610 2023-08-24T10:07:47.391 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:47.391 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":178,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:07:47.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.390+0000 7f5a5e26c640 1 -- 172.21.15.60:0/4166143400 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5a44037130 msgr2=0x7f5a44039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:47.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.390+0000 7f5a5e26c640 1 --2- 172.21.15.60:0/4166143400 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5a44037130 0x7f5a44039640 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:47.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.390+0000 7f5a5e26c640 1 -- 172.21.15.60:0/4166143400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a5812f610 msgr2=0x7f5a5809fca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:47.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.390+0000 7f5a5e26c640 1 --2- 172.21.15.60:0/4166143400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a5812f610 0x7f5a5809fca0 crc :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:47.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.390+0000 7f5a5e26c640 1 -- 172.21.15.60:0/4166143400 shutdown_connections 2023-08-24T10:07:47.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.390+0000 7f5a5e26c640 1 --2- 172.21.15.60:0/4166143400 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5a44037130 0x7f5a44039640 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:47.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.390+0000 7f5a5e26c640 1 --2- 172.21.15.60:0/4166143400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a5812f610 0x7f5a5809fca0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:47.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.390+0000 7f5a5e26c640 1 -- 172.21.15.60:0/4166143400 >> 172.21.15.60:0/4166143400 conn(0x7f5a580a2500 msgr2=0x7f5a581ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:47.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.390+0000 7f5a5e26c640 1 -- 172.21.15.60:0/4166143400 shutdown_connections 2023-08-24T10:07:47.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.390+0000 7f5a5e26c640 1 -- 172.21.15.60:0/4166143400 wait complete. 2023-08-24T10:07:47.403 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:07:47.403 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:47.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.509+0000 7f7b4eeac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:47.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.513+0000 7f7b4eeac640 1 Processor -- start 2023-08-24T10:07:47.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.513+0000 7f7b4eeac640 1 -- start start 2023-08-24T10:07:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.513+0000 7f7b4eeac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7b48135980 0x7f7b48137da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.513+0000 7f7b4eeac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b48132610 con 0x7f7b48132b50 2023-08-24T10:07:47.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.513+0000 7f7b4eeac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b4809c260 con 0x7f7b48135980 2023-08-24T10:07:47.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.513+0000 7f7b47fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7b48135980 0x7f7b48137da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:47.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.514+0000 7f7b47fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7b48135980 0x7f7b48137da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35530/0 (socket says 172.21.15.60:35530) 2023-08-24T10:07:47.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.514+0000 7f7b47fff640 1 -- 172.21.15.60:0/975465996 learned_addr learned my addr 172.21.15.60:0/975465996 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:47.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.514+0000 7f7b47fff640 1 -- 172.21.15.60:0/975465996 >> v1:172.21.15.60:6789/0 conn(0x7f7b48132b50 legacy=0x7f7b48132f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:47.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.514+0000 7f7b47fff640 1 -- 172.21.15.60:0/975465996 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b48132840 con 0x7f7b48135980 2023-08-24T10:07:47.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.514+0000 7f7b47fff640 1 --2- 172.21.15.60:0/975465996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b48135980 0x7f7b48137da0 crc :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c3eac163e385 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:47.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.514+0000 7f7b477fe640 1 -- 172.21.15.60:0/975465996 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7b38009dd0 con 0x7f7b48135980 2023-08-24T10:07:47.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.514+0000 7f7b477fe640 1 -- 172.21.15.60:0/975465996 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7b3800e230 con 0x7f7b48135980 2023-08-24T10:07:47.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.514+0000 7f7b477fe640 1 -- 172.21.15.60:0/975465996 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7b3800e4d0 con 0x7f7b48135980 2023-08-24T10:07:47.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.515+0000 7f7b4eeac640 1 -- 172.21.15.60:0/975465996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b48135980 msgr2=0x7f7b48137da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:47.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.515+0000 7f7b4eeac640 1 --2- 172.21.15.60:0/975465996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b48135980 0x7f7b48137da0 crc :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:47.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.515+0000 7f7b4eeac640 1 -- 172.21.15.60:0/975465996 shutdown_connections 2023-08-24T10:07:47.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.515+0000 7f7b4eeac640 1 --2- 172.21.15.60:0/975465996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b48135980 0x7f7b48137da0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:47.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.515+0000 7f7b4eeac640 1 -- 172.21.15.60:0/975465996 >> 172.21.15.60:0/975465996 conn(0x7f7b480a26a0 msgr2=0x7f7b480a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:47.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.515+0000 7f7b4eeac640 1 -- 172.21.15.60:0/975465996 shutdown_connections 2023-08-24T10:07:47.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.515+0000 7f7b4eeac640 1 -- 172.21.15.60:0/975465996 wait complete. 2023-08-24T10:07:47.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.515+0000 7f7b4eeac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:47.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.516+0000 7f7b4eeac640 1 Processor -- start 2023-08-24T10:07:47.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.516+0000 7f7b4eeac640 1 -- start start 2023-08-24T10:07:47.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.517+0000 7f7b4eeac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b48132b50 0x7f7b480a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:47.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.517+0000 7f7b4eeac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b480a1b60 con 0x7f7b48132b50 2023-08-24T10:07:47.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.517+0000 7f7b4cc48640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b48132b50 0x7f7b480a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:47.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.517+0000 7f7b4cc48640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b48132b50 0x7f7b480a0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35532/0 (socket says 172.21.15.60:35532) 2023-08-24T10:07:47.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.517+0000 7f7b4cc48640 1 -- 172.21.15.60:0/901626131 learned_addr learned my addr 172.21.15.60:0/901626131 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:47.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.517+0000 7f7b4cc48640 1 -- 172.21.15.60:0/901626131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4809fab0 con 0x7f7b48132b50 2023-08-24T10:07:47.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.517+0000 7f7b4cc48640 1 --2- 172.21.15.60:0/901626131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b48132b50 0x7f7b480a0000 crc :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:47.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.517+0000 7f7b457fa640 1 -- 172.21.15.60:0/901626131 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7b4000cbd0 con 0x7f7b48132b50 2023-08-24T10:07:47.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.517+0000 7f7b457fa640 1 -- 172.21.15.60:0/901626131 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7b4000cd30 con 0x7f7b48132b50 2023-08-24T10:07:47.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.517+0000 7f7b4eeac640 1 -- 172.21.15.60:0/901626131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4806a240 con 0x7f7b48132b50 2023-08-24T10:07:47.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.517+0000 7f7b457fa640 1 -- 172.21.15.60:0/901626131 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7b4000f230 con 0x7f7b48132b50 2023-08-24T10:07:47.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.517+0000 7f7b4eeac640 1 -- 172.21.15.60:0/901626131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4811ac50 con 0x7f7b48132b50 2023-08-24T10:07:47.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.518+0000 7f7b457fa640 1 -- 172.21.15.60:0/901626131 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f7b4000f3e0 con 0x7f7b48132b50 2023-08-24T10:07:47.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.518+0000 7f7b4eeac640 1 -- 172.21.15.60:0/901626131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b48061800 con 0x7f7b48132b50 2023-08-24T10:07:47.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.518+0000 7f7b457fa640 1 --2- 172.21.15.60:0/901626131 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7b28037130 0x7f7b28039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:47.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.518+0000 7f7b457fa640 1 -- 172.21.15.60:0/901626131 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f7b40046ba0 con 0x7f7b48132b50 2023-08-24T10:07:47.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.519+0000 7f7b47fff640 1 --2- 172.21.15.60:0/901626131 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7b28037130 0x7f7b28039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:47.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.519+0000 7f7b47fff640 1 --2- 172.21.15.60:0/901626131 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7b28037130 0x7f7b28039640 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:47.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.523+0000 7f7b457fa640 1 -- 172.21.15.60:0/901626131 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f7b4000ad00 con 0x7f7b48132b50 2023-08-24T10:07:47.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.698+0000 7f7b4eeac640 1 -- 172.21.15.60:0/901626131 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7b4809f940 con 0x7f7b28037130 2023-08-24T10:07:47.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.699+0000 7f7b457fa640 1 -- 172.21.15.60:0/901626131 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f7b4809f940 con 0x7f7b28037130 2023-08-24T10:07:47.701 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:47.702 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:47.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.701+0000 7f7b4eeac640 1 -- 172.21.15.60:0/901626131 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7b28037130 msgr2=0x7f7b28039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:47.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.701+0000 7f7b4eeac640 1 --2- 172.21.15.60:0/901626131 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7b28037130 0x7f7b28039640 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:47.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.701+0000 7f7b4eeac640 1 -- 172.21.15.60:0/901626131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b48132b50 msgr2=0x7f7b480a0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:47.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.701+0000 7f7b4eeac640 1 --2- 172.21.15.60:0/901626131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b48132b50 0x7f7b480a0000 crc :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:47.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.701+0000 7f7b4eeac640 1 -- 172.21.15.60:0/901626131 shutdown_connections 2023-08-24T10:07:47.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.701+0000 7f7b4eeac640 1 --2- 172.21.15.60:0/901626131 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7b28037130 0x7f7b28039640 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:47.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.701+0000 7f7b4eeac640 1 --2- 172.21.15.60:0/901626131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b48132b50 0x7f7b480a0000 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:47.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.701+0000 7f7b4eeac640 1 -- 172.21.15.60:0/901626131 >> 172.21.15.60:0/901626131 conn(0x7f7b480a26a0 msgr2=0x7f7b481baaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:47.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.701+0000 7f7b4eeac640 1 -- 172.21.15.60:0/901626131 shutdown_connections 2023-08-24T10:07:47.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:47.701+0000 7f7b4eeac640 1 -- 172.21.15.60:0/901626131 wait complete. 2023-08-24T10:07:47.713 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2023-08-24T10:07:47.528237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:48.770 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~161s 2023-08-24T10:07:48.770 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~161s 2023-08-24T10:07:48.770 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~161s 2023-08-24T10:07:48.770 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~161s 2023-08-24T10:07:50.715 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:50.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.835+0000 7efd23aec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:50.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.839+0000 7efd23aec640 1 Processor -- start 2023-08-24T10:07:50.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.839+0000 7efd23aec640 1 -- start start 2023-08-24T10:07:50.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.840+0000 7efd23aec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efd1c096860 0x7efd1c096c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:50.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.840+0000 7efd23aec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1c0971b0 con 0x7efd1c0990b0 2023-08-24T10:07:50.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.840+0000 7efd23aec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd1c06db70 con 0x7efd1c096860 2023-08-24T10:07:50.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.840+0000 7efd21888640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efd1c096860 0x7efd1c096c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:50.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.840+0000 7efd21888640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efd1c096860 0x7efd1c096c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35536/0 (socket says 172.21.15.60:35536) 2023-08-24T10:07:50.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.840+0000 7efd21087640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efd1c0990b0 0x7efd1c0994c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54226/0 (socket says 172.21.15.60:54226) 2023-08-24T10:07:50.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.840+0000 7efd21888640 1 -- 172.21.15.60:0/4076807985 learned_addr learned my addr 172.21.15.60:0/4076807985 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:50.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.840+0000 7efd21888640 1 -- 172.21.15.60:0/4076807985 >> v1:172.21.15.60:6789/0 conn(0x7efd1c0990b0 legacy=0x7efd1c0994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:50.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.841+0000 7efd21888640 1 -- 172.21.15.60:0/4076807985 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c06e320 con 0x7efd1c096860 2023-08-24T10:07:50.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.841+0000 7efd21888640 1 --2- 172.21.15.60:0/4076807985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd1c096860 0x7efd1c096c70 crc :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5242c353f169ce7 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:50.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.841+0000 7efd20886640 1 -- 172.21.15.60:0/4076807985 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efd18009dd0 con 0x7efd1c096860 2023-08-24T10:07:50.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.841+0000 7efd20886640 1 -- 172.21.15.60:0/4076807985 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7efd1800e230 con 0x7efd1c096860 2023-08-24T10:07:50.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.841+0000 7efd20886640 1 -- 172.21.15.60:0/4076807985 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efd1800e4d0 con 0x7efd1c096860 2023-08-24T10:07:50.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.841+0000 7efd23aec640 1 -- 172.21.15.60:0/4076807985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd1c096860 msgr2=0x7efd1c096c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:50.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.841+0000 7efd23aec640 1 --2- 172.21.15.60:0/4076807985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd1c096860 0x7efd1c096c70 crc :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:50.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.842+0000 7efd23aec640 1 -- 172.21.15.60:0/4076807985 shutdown_connections 2023-08-24T10:07:50.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.842+0000 7efd23aec640 1 --2- 172.21.15.60:0/4076807985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd1c096860 0x7efd1c096c70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:50.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.842+0000 7efd23aec640 1 -- 172.21.15.60:0/4076807985 >> 172.21.15.60:0/4076807985 conn(0x7efd1c09e110 msgr2=0x7efd1c09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:50.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.842+0000 7efd23aec640 1 -- 172.21.15.60:0/4076807985 shutdown_connections 2023-08-24T10:07:50.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.842+0000 7efd23aec640 1 -- 172.21.15.60:0/4076807985 wait complete. 2023-08-24T10:07:50.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.842+0000 7efd23aec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:50.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.843+0000 7efd23aec640 1 Processor -- start 2023-08-24T10:07:50.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.843+0000 7efd23aec640 1 -- start start 2023-08-24T10:07:50.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.843+0000 7efd23aec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd1c096860 0x7efd1c126ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:50.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.843+0000 7efd23aec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c130220 con 0x7efd1c096860 2023-08-24T10:07:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.844+0000 7efd21888640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd1c096860 0x7efd1c126ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.844+0000 7efd21888640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd1c096860 0x7efd1c126ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35546/0 (socket says 172.21.15.60:35546) 2023-08-24T10:07:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.844+0000 7efd21888640 1 -- 172.21.15.60:0/1024684645 learned_addr learned my addr 172.21.15.60:0/1024684645 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:50.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.844+0000 7efd21888640 1 -- 172.21.15.60:0/1024684645 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c128a90 con 0x7efd1c096860 2023-08-24T10:07:50.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.844+0000 7efd21888640 1 --2- 172.21.15.60:0/1024684645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd1c096860 0x7efd1c126ef0 crc :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:50.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.844+0000 7efd127fc640 1 -- 172.21.15.60:0/1024684645 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efd18014540 con 0x7efd1c096860 2023-08-24T10:07:50.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.844+0000 7efd127fc640 1 -- 172.21.15.60:0/1024684645 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7efd180146a0 con 0x7efd1c096860 2023-08-24T10:07:50.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.844+0000 7efd23aec640 1 -- 172.21.15.60:0/1024684645 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1c12ff10 con 0x7efd1c096860 2023-08-24T10:07:50.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.844+0000 7efd127fc640 1 -- 172.21.15.60:0/1024684645 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efd18014940 con 0x7efd1c096860 2023-08-24T10:07:50.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.844+0000 7efd23aec640 1 -- 172.21.15.60:0/1024684645 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1c12a370 con 0x7efd1c096860 2023-08-24T10:07:50.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.845+0000 7efd23aec640 1 -- 172.21.15.60:0/1024684645 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd1c12ade0 con 0x7efd1c096860 2023-08-24T10:07:50.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.845+0000 7efd127fc640 1 -- 172.21.15.60:0/1024684645 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7efd18014af0 con 0x7efd1c096860 2023-08-24T10:07:50.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.845+0000 7efd127fc640 1 --2- 172.21.15.60:0/1024684645 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efd00037130 0x7efd00039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:50.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.845+0000 7efd127fc640 1 -- 172.21.15.60:0/1024684645 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7efd18019040 con 0x7efd1c096860 2023-08-24T10:07:50.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.846+0000 7efd21087640 1 --2- 172.21.15.60:0/1024684645 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efd00037130 0x7efd00039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:50.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.846+0000 7efd21087640 1 --2- 172.21.15.60:0/1024684645 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efd00037130 0x7efd00039640 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:50.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:50.850+0000 7efd127fc640 1 -- 172.21.15.60:0/1024684645 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7efd1c12ade0 con 0x7efd1c096860 2023-08-24T10:07:51.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.024+0000 7efd23aec640 1 -- 172.21.15.60:0/1024684645 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd1c06d190 con 0x7efd00037130 2023-08-24T10:07:51.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.026+0000 7efd127fc640 1 -- 172.21.15.60:0/1024684645 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7efd0c0070b0 con 0x7efd00037130 2023-08-24T10:07:51.028 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:51.029 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:51.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.027+0000 7efd23aec640 1 -- 172.21.15.60:0/1024684645 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efd00037130 msgr2=0x7efd00039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:51.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.027+0000 7efd23aec640 1 --2- 172.21.15.60:0/1024684645 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efd00037130 0x7efd00039640 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:51.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.027+0000 7efd23aec640 1 -- 172.21.15.60:0/1024684645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd1c096860 msgr2=0x7efd1c126ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:51.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.027+0000 7efd23aec640 1 --2- 172.21.15.60:0/1024684645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd1c096860 0x7efd1c126ef0 crc :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:51.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.027+0000 7efd23aec640 1 -- 172.21.15.60:0/1024684645 shutdown_connections 2023-08-24T10:07:51.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.027+0000 7efd23aec640 1 --2- 172.21.15.60:0/1024684645 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efd00037130 0x7efd00039640 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.027+0000 7efd23aec640 1 --2- 172.21.15.60:0/1024684645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd1c096860 0x7efd1c126ef0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.027+0000 7efd23aec640 1 -- 172.21.15.60:0/1024684645 >> 172.21.15.60:0/1024684645 conn(0x7efd1c09e110 msgr2=0x7efd1c1b73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.028+0000 7efd23aec640 1 -- 172.21.15.60:0/1024684645 shutdown_connections 2023-08-24T10:07:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.028+0000 7efd23aec640 1 -- 172.21.15.60:0/1024684645 wait complete. 2023-08-24T10:07:51.040 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2023-08-24T10:07:49.528715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:51.041 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:51.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.147+0000 7fe9fa11e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:51.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.151+0000 7fe9fa11e640 1 Processor -- start 2023-08-24T10:07:51.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.151+0000 7fe9fa11e640 1 -- start start 2023-08-24T10:07:51.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.152+0000 7fe9fa11e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe9f406d300 0x7fe9f412bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:51.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.152+0000 7fe9fa11e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f4099440 con 0x7fe9f412c1c0 2023-08-24T10:07:51.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.152+0000 7fe9fa11e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f409a050 con 0x7fe9f406d300 2023-08-24T10:07:51.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.152+0000 7fe9f37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe9f406d300 0x7fe9f412bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:51.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.152+0000 7fe9f37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe9f406d300 0x7fe9f412bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35558/0 (socket says 172.21.15.60:35558) 2023-08-24T10:07:51.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.152+0000 7fe9f37fe640 1 -- 172.21.15.60:0/2836223578 learned_addr learned my addr 172.21.15.60:0/2836223578 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:51.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.152+0000 7fe9f2ffd640 1 --1- 172.21.15.60:0/2836223578 >> v1:172.21.15.60:6789/0 conn(0x7fe9f412c1c0 0x7fe9f412ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54242/0 (socket says 172.21.15.60:54242) 2023-08-24T10:07:51.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.152+0000 7fe9f37fe640 1 -- 172.21.15.60:0/2836223578 >> v1:172.21.15.60:6789/0 conn(0x7fe9f412c1c0 legacy=0x7fe9f412ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:51.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.152+0000 7fe9f37fe640 1 -- 172.21.15.60:0/2836223578 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f412b9d0 con 0x7fe9f406d300 2023-08-24T10:07:51.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.153+0000 7fe9f37fe640 1 --2- 172.21.15.60:0/2836223578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f406d300 0x7fe9f412bc80 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79126f256e3615c6 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:51.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.153+0000 7fe9f27fc640 1 -- 172.21.15.60:0/2836223578 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe9e4009dd0 con 0x7fe9f406d300 2023-08-24T10:07:51.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.153+0000 7fe9f27fc640 1 -- 172.21.15.60:0/2836223578 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fe9e400e230 con 0x7fe9f406d300 2023-08-24T10:07:51.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.153+0000 7fe9f27fc640 1 -- 172.21.15.60:0/2836223578 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe9e400e4d0 con 0x7fe9f406d300 2023-08-24T10:07:51.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.153+0000 7fe9fa11e640 1 -- 172.21.15.60:0/2836223578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f406d300 msgr2=0x7fe9f412bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:51.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.153+0000 7fe9fa11e640 1 --2- 172.21.15.60:0/2836223578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f406d300 0x7fe9f412bc80 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:51.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.153+0000 7fe9fa11e640 1 -- 172.21.15.60:0/2836223578 shutdown_connections 2023-08-24T10:07:51.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.153+0000 7fe9fa11e640 1 --2- 172.21.15.60:0/2836223578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f406d300 0x7fe9f412bc80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:51.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.153+0000 7fe9fa11e640 1 -- 172.21.15.60:0/2836223578 >> 172.21.15.60:0/2836223578 conn(0x7fe9f409e110 msgr2=0x7fe9f409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:51.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.153+0000 7fe9fa11e640 1 -- 172.21.15.60:0/2836223578 shutdown_connections 2023-08-24T10:07:51.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.154+0000 7fe9fa11e640 1 -- 172.21.15.60:0/2836223578 wait complete. 2023-08-24T10:07:51.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.154+0000 7fe9fa11e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:51.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.155+0000 7fe9fa11e640 1 Processor -- start 2023-08-24T10:07:51.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.155+0000 7fe9fa11e640 1 -- start start 2023-08-24T10:07:51.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.155+0000 7fe9fa11e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f406d300 0x7fe9f4070f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:51.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.155+0000 7fe9fa11e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f40705e0 con 0x7fe9f406d300 2023-08-24T10:07:51.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.156+0000 7fe9f37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f406d300 0x7fe9f4070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:51.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.156+0000 7fe9f37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f406d300 0x7fe9f4070f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35562/0 (socket says 172.21.15.60:35562) 2023-08-24T10:07:51.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.156+0000 7fe9f37fe640 1 -- 172.21.15.60:0/295837620 learned_addr learned my addr 172.21.15.60:0/295837620 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:51.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.156+0000 7fe9f37fe640 1 -- 172.21.15.60:0/295837620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f4070b70 con 0x7fe9f406d300 2023-08-24T10:07:51.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.156+0000 7fe9f37fe640 1 --2- 172.21.15.60:0/295837620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f406d300 0x7fe9f4070f30 crc :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:51.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.156+0000 7fe9dbfff640 1 -- 172.21.15.60:0/295837620 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe9e4014540 con 0x7fe9f406d300 2023-08-24T10:07:51.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.156+0000 7fe9dbfff640 1 -- 172.21.15.60:0/295837620 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fe9e40146a0 con 0x7fe9f406d300 2023-08-24T10:07:51.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.156+0000 7fe9fa11e640 1 -- 172.21.15.60:0/295837620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f4136af0 con 0x7fe9f406d300 2023-08-24T10:07:51.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.156+0000 7fe9dbfff640 1 -- 172.21.15.60:0/295837620 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe9e4014940 con 0x7fe9f406d300 2023-08-24T10:07:51.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.156+0000 7fe9fa11e640 1 -- 172.21.15.60:0/295837620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f4132440 con 0x7fe9f406d300 2023-08-24T10:07:51.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.157+0000 7fe9dbfff640 1 -- 172.21.15.60:0/295837620 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fe9e4014af0 con 0x7fe9f406d300 2023-08-24T10:07:51.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.157+0000 7fe9fa11e640 1 -- 172.21.15.60:0/295837620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f4061800 con 0x7fe9f406d300 2023-08-24T10:07:51.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.157+0000 7fe9dbfff640 1 --2- 172.21.15.60:0/295837620 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe9e0037130 0x7fe9e0039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:51.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.157+0000 7fe9dbfff640 1 -- 172.21.15.60:0/295837620 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fe9e4019040 con 0x7fe9f406d300 2023-08-24T10:07:51.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.158+0000 7fe9f2ffd640 1 --2- 172.21.15.60:0/295837620 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe9e0037130 0x7fe9e0039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:51.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.158+0000 7fe9f2ffd640 1 --2- 172.21.15.60:0/295837620 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe9e0037130 0x7fe9e0039640 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:51.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.162+0000 7fe9dbfff640 1 -- 172.21.15.60:0/295837620 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fe9e4018190 con 0x7fe9f406d300 2023-08-24T10:07:51.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.333+0000 7fe9fa11e640 1 -- 172.21.15.60:0/295837620 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe9f4070970 con 0x7fe9e0037130 2023-08-24T10:07:51.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.335+0000 7fe9dbfff640 1 -- 172.21.15.60:0/295837620 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7fe9f4070970 con 0x7fe9e0037130 2023-08-24T10:07:51.337 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:51.337 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:51.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.336+0000 7fe9fa11e640 1 -- 172.21.15.60:0/295837620 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe9e0037130 msgr2=0x7fe9e0039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:51.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.336+0000 7fe9fa11e640 1 --2- 172.21.15.60:0/295837620 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe9e0037130 0x7fe9e0039640 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:51.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.336+0000 7fe9fa11e640 1 -- 172.21.15.60:0/295837620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f406d300 msgr2=0x7fe9f4070f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:51.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.336+0000 7fe9fa11e640 1 --2- 172.21.15.60:0/295837620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f406d300 0x7fe9f4070f30 crc :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:51.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.336+0000 7fe9fa11e640 1 -- 172.21.15.60:0/295837620 shutdown_connections 2023-08-24T10:07:51.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.336+0000 7fe9fa11e640 1 --2- 172.21.15.60:0/295837620 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe9e0037130 0x7fe9e0039640 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:51.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.336+0000 7fe9fa11e640 1 --2- 172.21.15.60:0/295837620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9f406d300 0x7fe9f4070f30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:51.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.336+0000 7fe9fa11e640 1 -- 172.21.15.60:0/295837620 >> 172.21.15.60:0/295837620 conn(0x7fe9f409e110 msgr2=0x7fe9f40978f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:51.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.337+0000 7fe9fa11e640 1 -- 172.21.15.60:0/295837620 shutdown_connections 2023-08-24T10:07:51.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.337+0000 7fe9fa11e640 1 -- 172.21.15.60:0/295837620 wait complete. 2023-08-24T10:07:51.349 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2023-08-24T10:07:49.528715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:51.350 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:07:51.350 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:51.350 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:07:51.351 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:51.351 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:07:51.351 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:51.351 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:07:51.352 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:51.352 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:07:51.352 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:51.352 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:07:51.353 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:51.353 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:07:51.353 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:51.353 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:07:51.354 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:51.354 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:07:51.354 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:51.354 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:07:51.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.461+0000 7f36a74ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:51.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.464+0000 7f36a74ac640 1 Processor -- start 2023-08-24T10:07:51.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.464+0000 7f36a74ac640 1 -- start start 2023-08-24T10:07:51.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.465+0000 7f36a74ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f36a0070c30 0x7f36a0071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:51.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.465+0000 7f36a74ac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a006dfc0 con 0x7f36a00990b0 2023-08-24T10:07:51.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.465+0000 7f36a74ac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a006d9b0 con 0x7f36a0070c30 2023-08-24T10:07:51.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.465+0000 7f36a5248640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f36a0070c30 0x7f36a0071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:51.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.465+0000 7f36a5248640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f36a0070c30 0x7f36a0071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35574/0 (socket says 172.21.15.60:35574) 2023-08-24T10:07:51.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.465+0000 7f36a5248640 1 -- 172.21.15.60:0/2846380604 learned_addr learned my addr 172.21.15.60:0/2846380604 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:51.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.465+0000 7f36a4a47640 1 --1- 172.21.15.60:0/2846380604 >> v1:172.21.15.60:6789/0 conn(0x7f36a00990b0 0x7f36a00994c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54258/0 (socket says 172.21.15.60:54258) 2023-08-24T10:07:51.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.465+0000 7f36a5248640 1 -- 172.21.15.60:0/2846380604 >> v1:172.21.15.60:6789/0 conn(0x7f36a00990b0 legacy=0x7f36a00994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:51.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.465+0000 7f36a5248640 1 -- 172.21.15.60:0/2846380604 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a006e6e0 con 0x7f36a0070c30 2023-08-24T10:07:51.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.465+0000 7f36a5248640 1 --2- 172.21.15.60:0/2846380604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0070c30 0x7f36a0071040 crc :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d62f17e2044fe4 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:51.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.465+0000 7f3697fff640 1 -- 172.21.15.60:0/2846380604 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f369c010570 con 0x7f36a0070c30 2023-08-24T10:07:51.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.465+0000 7f3697fff640 1 -- 172.21.15.60:0/2846380604 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f369c0106d0 con 0x7f36a0070c30 2023-08-24T10:07:51.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.465+0000 7f3697fff640 1 -- 172.21.15.60:0/2846380604 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f369c010970 con 0x7f36a0070c30 2023-08-24T10:07:51.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.466+0000 7f36a74ac640 1 -- 172.21.15.60:0/2846380604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0070c30 msgr2=0x7f36a0071040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:51.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.466+0000 7f36a74ac640 1 --2- 172.21.15.60:0/2846380604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0070c30 0x7f36a0071040 crc :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:51.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.466+0000 7f36a74ac640 1 -- 172.21.15.60:0/2846380604 shutdown_connections 2023-08-24T10:07:51.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.466+0000 7f36a74ac640 1 --2- 172.21.15.60:0/2846380604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0070c30 0x7f36a0071040 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:51.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.466+0000 7f36a74ac640 1 -- 172.21.15.60:0/2846380604 >> 172.21.15.60:0/2846380604 conn(0x7f36a009e110 msgr2=0x7f36a009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:51.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.466+0000 7f36a74ac640 1 -- 172.21.15.60:0/2846380604 shutdown_connections 2023-08-24T10:07:51.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.466+0000 7f36a74ac640 1 -- 172.21.15.60:0/2846380604 wait complete. 2023-08-24T10:07:51.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.466+0000 7f36a74ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:51.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.467+0000 7f36a74ac640 1 Processor -- start 2023-08-24T10:07:51.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.467+0000 7f36a74ac640 1 -- start start 2023-08-24T10:07:51.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.468+0000 7f36a74ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0070c30 0x7f36a01c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:51.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.468+0000 7f36a74ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a012dec0 con 0x7f36a0070c30 2023-08-24T10:07:51.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.468+0000 7f36a5248640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0070c30 0x7f36a01c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:51.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.468+0000 7f36a5248640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0070c30 0x7f36a01c31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35582/0 (socket says 172.21.15.60:35582) 2023-08-24T10:07:51.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.468+0000 7f36a5248640 1 -- 172.21.15.60:0/1758918090 learned_addr learned my addr 172.21.15.60:0/1758918090 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:51.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.468+0000 7f36a5248640 1 -- 172.21.15.60:0/1758918090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a012b9d0 con 0x7f36a0070c30 2023-08-24T10:07:51.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.468+0000 7f36a5248640 1 --2- 172.21.15.60:0/1758918090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0070c30 0x7f36a01c31e0 crc :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:51.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.469+0000 7f3695ffb640 1 -- 172.21.15.60:0/1758918090 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f369c00f8f0 con 0x7f36a0070c30 2023-08-24T10:07:51.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.469+0000 7f3695ffb640 1 -- 172.21.15.60:0/1758918090 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f369c016040 con 0x7f36a0070c30 2023-08-24T10:07:51.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.469+0000 7f3695ffb640 1 -- 172.21.15.60:0/1758918090 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f369c00fb80 con 0x7f36a0070c30 2023-08-24T10:07:51.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.469+0000 7f36a74ac640 1 -- 172.21.15.60:0/1758918090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a012ca70 con 0x7f36a0070c30 2023-08-24T10:07:51.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.469+0000 7f36a74ac640 1 -- 172.21.15.60:0/1758918090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a012d210 con 0x7f36a0070c30 2023-08-24T10:07:51.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.470+0000 7f3695ffb640 1 -- 172.21.15.60:0/1758918090 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f369c00fd80 con 0x7f36a0070c30 2023-08-24T10:07:51.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.470+0000 7f36a74ac640 1 -- 172.21.15.60:0/1758918090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a006e220 con 0x7f36a0070c30 2023-08-24T10:07:51.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.470+0000 7f3695ffb640 1 --2- 172.21.15.60:0/1758918090 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f36840370e0 0x7f36840395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:51.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.470+0000 7f3695ffb640 1 -- 172.21.15.60:0/1758918090 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f369c04a310 con 0x7f36a0070c30 2023-08-24T10:07:51.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.470+0000 7f36a4a47640 1 --2- 172.21.15.60:0/1758918090 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f36840370e0 0x7f36840395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:51.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.471+0000 7f36a4a47640 1 --2- 172.21.15.60:0/1758918090 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f36840370e0 0x7f36840395f0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:51.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.474+0000 7f3695ffb640 1 -- 172.21.15.60:0/1758918090 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f369c018a40 con 0x7f36a0070c30 2023-08-24T10:07:51.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.720+0000 7f36a74ac640 1 -- 172.21.15.60:0/1758918090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f36a0061800 con 0x7f36a0070c30 2023-08-24T10:07:51.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.720+0000 7f3695ffb640 1 -- 172.21.15.60:0/1758918090 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f369c00c2e0 con 0x7f36a0070c30 2023-08-24T10:07:51.723 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:51.723 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":182,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:07:51.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.722+0000 7f36a74ac640 1 -- 172.21.15.60:0/1758918090 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f36840370e0 msgr2=0x7f36840395f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:51.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.722+0000 7f36a74ac640 1 --2- 172.21.15.60:0/1758918090 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f36840370e0 0x7f36840395f0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:51.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.722+0000 7f36a74ac640 1 -- 172.21.15.60:0/1758918090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0070c30 msgr2=0x7f36a01c31e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:51.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.722+0000 7f36a74ac640 1 --2- 172.21.15.60:0/1758918090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0070c30 0x7f36a01c31e0 crc :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:51.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.722+0000 7f36a74ac640 1 -- 172.21.15.60:0/1758918090 shutdown_connections 2023-08-24T10:07:51.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.722+0000 7f36a74ac640 1 --2- 172.21.15.60:0/1758918090 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f36840370e0 0x7f36840395f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:51.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.722+0000 7f36a74ac640 1 --2- 172.21.15.60:0/1758918090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0070c30 0x7f36a01c31e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:51.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.722+0000 7f36a74ac640 1 -- 172.21.15.60:0/1758918090 >> 172.21.15.60:0/1758918090 conn(0x7f36a009e110 msgr2=0x7f36a009ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:51.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.722+0000 7f36a74ac640 1 -- 172.21.15.60:0/1758918090 shutdown_connections 2023-08-24T10:07:51.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.722+0000 7f36a74ac640 1 -- 172.21.15.60:0/1758918090 wait complete. 2023-08-24T10:07:51.735 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:07:51.736 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:51.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.843+0000 7fe6be95e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:51.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.846+0000 7fe6be95e640 1 Processor -- start 2023-08-24T10:07:51.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.847+0000 7fe6be95e640 1 -- start start 2023-08-24T10:07:51.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.847+0000 7fe6be95e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe6b806d300 0x7fe6b812bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:51.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.847+0000 7fe6be95e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b8099440 con 0x7fe6b812c1c0 2023-08-24T10:07:51.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.847+0000 7fe6be95e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b809a050 con 0x7fe6b806d300 2023-08-24T10:07:51.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.847+0000 7fe6b7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe6b806d300 0x7fe6b812bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:51.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.847+0000 7fe6b7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe6b806d300 0x7fe6b812bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35594/0 (socket says 172.21.15.60:35594) 2023-08-24T10:07:51.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.847+0000 7fe6b7fff640 1 -- 172.21.15.60:0/2757947001 learned_addr learned my addr 172.21.15.60:0/2757947001 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:51.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.847+0000 7fe6b77fe640 1 --1- 172.21.15.60:0/2757947001 >> v1:172.21.15.60:6789/0 conn(0x7fe6b812c1c0 0x7fe6b812ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54274/0 (socket says 172.21.15.60:54274) 2023-08-24T10:07:51.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.848+0000 7fe6b7fff640 1 -- 172.21.15.60:0/2757947001 >> v1:172.21.15.60:6789/0 conn(0x7fe6b812c1c0 legacy=0x7fe6b812ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:51.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.848+0000 7fe6b7fff640 1 -- 172.21.15.60:0/2757947001 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b812b9d0 con 0x7fe6b806d300 2023-08-24T10:07:51.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.848+0000 7fe6b7fff640 1 --2- 172.21.15.60:0/2757947001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6b806d300 0x7fe6b812bc80 crc :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af42735eaa4175c7 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:51.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.848+0000 7fe6b6ffd640 1 -- 172.21.15.60:0/2757947001 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe6ac009dd0 con 0x7fe6b806d300 2023-08-24T10:07:51.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.848+0000 7fe6b6ffd640 1 -- 172.21.15.60:0/2757947001 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fe6ac00e230 con 0x7fe6b806d300 2023-08-24T10:07:51.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.848+0000 7fe6b6ffd640 1 -- 172.21.15.60:0/2757947001 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe6ac00e4d0 con 0x7fe6b806d300 2023-08-24T10:07:51.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.849+0000 7fe6be95e640 1 -- 172.21.15.60:0/2757947001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6b806d300 msgr2=0x7fe6b812bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:51.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.849+0000 7fe6be95e640 1 --2- 172.21.15.60:0/2757947001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6b806d300 0x7fe6b812bc80 crc :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:51.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.849+0000 7fe6be95e640 1 -- 172.21.15.60:0/2757947001 shutdown_connections 2023-08-24T10:07:51.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.849+0000 7fe6be95e640 1 --2- 172.21.15.60:0/2757947001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6b806d300 0x7fe6b812bc80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:51.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.849+0000 7fe6be95e640 1 -- 172.21.15.60:0/2757947001 >> 172.21.15.60:0/2757947001 conn(0x7fe6b809e110 msgr2=0x7fe6b809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:51.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.849+0000 7fe6be95e640 1 -- 172.21.15.60:0/2757947001 shutdown_connections 2023-08-24T10:07:51.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.849+0000 7fe6be95e640 1 -- 172.21.15.60:0/2757947001 wait complete. 2023-08-24T10:07:51.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.849+0000 7fe6be95e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:51.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.850+0000 7fe6be95e640 1 Processor -- start 2023-08-24T10:07:51.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.850+0000 7fe6be95e640 1 -- start start 2023-08-24T10:07:51.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.851+0000 7fe6be95e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6b806d300 0x7fe6b80964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:51.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.851+0000 7fe6be95e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b81304d0 con 0x7fe6b806d300 2023-08-24T10:07:51.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.851+0000 7fe6b7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6b806d300 0x7fe6b80964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:51.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.851+0000 7fe6b7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6b806d300 0x7fe6b80964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35606/0 (socket says 172.21.15.60:35606) 2023-08-24T10:07:51.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.851+0000 7fe6b7fff640 1 -- 172.21.15.60:0/3669955327 learned_addr learned my addr 172.21.15.60:0/3669955327 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:51.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.851+0000 7fe6b7fff640 1 -- 172.21.15.60:0/3669955327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b81301c0 con 0x7fe6b806d300 2023-08-24T10:07:51.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.851+0000 7fe6b7fff640 1 --2- 172.21.15.60:0/3669955327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6b806d300 0x7fe6b80964a0 crc :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:51.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.851+0000 7fe6b4ff9640 1 -- 172.21.15.60:0/3669955327 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe6ac014540 con 0x7fe6b806d300 2023-08-24T10:07:51.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.851+0000 7fe6b4ff9640 1 -- 172.21.15.60:0/3669955327 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fe6ac0146a0 con 0x7fe6b806d300 2023-08-24T10:07:51.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.851+0000 7fe6be95e640 1 -- 172.21.15.60:0/3669955327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b80715e0 con 0x7fe6b806d300 2023-08-24T10:07:51.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.851+0000 7fe6b4ff9640 1 -- 172.21.15.60:0/3669955327 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe6ac014940 con 0x7fe6b806d300 2023-08-24T10:07:51.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.851+0000 7fe6be95e640 1 -- 172.21.15.60:0/3669955327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b81359f0 con 0x7fe6b806d300 2023-08-24T10:07:51.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.852+0000 7fe6b4ff9640 1 -- 172.21.15.60:0/3669955327 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fe6ac014af0 con 0x7fe6b806d300 2023-08-24T10:07:51.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.852+0000 7fe6be95e640 1 -- 172.21.15.60:0/3669955327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b806d9f0 con 0x7fe6b806d300 2023-08-24T10:07:51.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.853+0000 7fe6b4ff9640 1 --2- 172.21.15.60:0/3669955327 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe6a4037130 0x7fe6a4039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:51.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.853+0000 7fe6b4ff9640 1 -- 172.21.15.60:0/3669955327 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fe6ac019040 con 0x7fe6b806d300 2023-08-24T10:07:51.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.853+0000 7fe6b77fe640 1 --2- 172.21.15.60:0/3669955327 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe6a4037130 0x7fe6a4039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:51.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.853+0000 7fe6b77fe640 1 --2- 172.21.15.60:0/3669955327 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe6a4037130 0x7fe6a4039640 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:51.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:51.857+0000 7fe6b4ff9640 1 -- 172.21.15.60:0/3669955327 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fe6ac018190 con 0x7fe6b806d300 2023-08-24T10:07:52.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:52.029+0000 7fe6be95e640 1 -- 172.21.15.60:0/3669955327 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe6b809bc70 con 0x7fe6a4037130 2023-08-24T10:07:52.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:52.031+0000 7fe6b4ff9640 1 -- 172.21.15.60:0/3669955327 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7fe6a80070b0 con 0x7fe6a4037130 2023-08-24T10:07:52.032 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:52.033 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:52.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:52.032+0000 7fe6be95e640 1 -- 172.21.15.60:0/3669955327 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe6a4037130 msgr2=0x7fe6a4039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:52.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:52.032+0000 7fe6be95e640 1 --2- 172.21.15.60:0/3669955327 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe6a4037130 0x7fe6a4039640 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:52.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:52.032+0000 7fe6be95e640 1 -- 172.21.15.60:0/3669955327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6b806d300 msgr2=0x7fe6b80964a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:52.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:52.032+0000 7fe6be95e640 1 --2- 172.21.15.60:0/3669955327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6b806d300 0x7fe6b80964a0 crc :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:52.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:52.032+0000 7fe6be95e640 1 -- 172.21.15.60:0/3669955327 shutdown_connections 2023-08-24T10:07:52.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:52.032+0000 7fe6be95e640 1 --2- 172.21.15.60:0/3669955327 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe6a4037130 0x7fe6a4039640 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:52.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:52.032+0000 7fe6be95e640 1 --2- 172.21.15.60:0/3669955327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6b806d300 0x7fe6b80964a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:52.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:52.032+0000 7fe6be95e640 1 -- 172.21.15.60:0/3669955327 >> 172.21.15.60:0/3669955327 conn(0x7fe6b809e110 msgr2=0x7fe6b812e820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:52.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:52.032+0000 7fe6be95e640 1 -- 172.21.15.60:0/3669955327 shutdown_connections 2023-08-24T10:07:52.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:52.032+0000 7fe6be95e640 1 -- 172.21.15.60:0/3669955327 wait complete. 2023-08-24T10:07:52.045 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2023-08-24T10:07:51.528976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:53.973 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~167s 2023-08-24T10:07:53.973 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~167s 2023-08-24T10:07:53.973 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~167s 2023-08-24T10:07:53.973 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~167s 2023-08-24T10:07:55.048 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:55.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.168+0000 7fcfec4ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:55.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.172+0000 7fcfec4ac640 1 Processor -- start 2023-08-24T10:07:55.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.172+0000 7fcfec4ac640 1 -- start start 2023-08-24T10:07:55.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.173+0000 7fcfec4ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcfe4132630 0x7fcfe4134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:55.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.173+0000 7fcfec4ac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe406e320 con 0x7fcfe4096860 2023-08-24T10:07:55.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.173+0000 7fcfec4ac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe406db70 con 0x7fcfe4132630 2023-08-24T10:07:55.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.173+0000 7fcfe9a47640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcfe4132630 0x7fcfe4134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:55.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.173+0000 7fcfe9a47640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcfe4132630 0x7fcfe4134a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35612/0 (socket says 172.21.15.60:35612) 2023-08-24T10:07:55.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.173+0000 7fcfe9a47640 1 -- 172.21.15.60:0/1505305538 learned_addr learned my addr 172.21.15.60:0/1505305538 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:55.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.173+0000 7fcfe9a47640 1 -- 172.21.15.60:0/1505305538 >> v1:172.21.15.60:6789/0 conn(0x7fcfe4096860 legacy=0x7fcfe4096c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:55.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.174+0000 7fcfe9a47640 1 -- 172.21.15.60:0/1505305538 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe406cef0 con 0x7fcfe4132630 2023-08-24T10:07:55.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.174+0000 7fcfe9a47640 1 --2- 172.21.15.60:0/1505305538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfe4132630 0x7fcfe4134a50 crc :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4941b2b2cb7050d6 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:55.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.174+0000 7fcfe9246640 1 -- 172.21.15.60:0/1505305538 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcfd4009dd0 con 0x7fcfe4132630 2023-08-24T10:07:55.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.174+0000 7fcfe9246640 1 -- 172.21.15.60:0/1505305538 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fcfd400e230 con 0x7fcfe4132630 2023-08-24T10:07:55.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.174+0000 7fcfe9246640 1 -- 172.21.15.60:0/1505305538 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcfd400e4d0 con 0x7fcfe4132630 2023-08-24T10:07:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.174+0000 7fcfec4ac640 1 -- 172.21.15.60:0/1505305538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfe4132630 msgr2=0x7fcfe4134a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.174+0000 7fcfec4ac640 1 --2- 172.21.15.60:0/1505305538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfe4132630 0x7fcfe4134a50 crc :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.175+0000 7fcfec4ac640 1 -- 172.21.15.60:0/1505305538 shutdown_connections 2023-08-24T10:07:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.175+0000 7fcfec4ac640 1 --2- 172.21.15.60:0/1505305538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfe4132630 0x7fcfe4134a50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.175+0000 7fcfec4ac640 1 -- 172.21.15.60:0/1505305538 >> 172.21.15.60:0/1505305538 conn(0x7fcfe409e110 msgr2=0x7fcfe409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.175+0000 7fcfec4ac640 1 -- 172.21.15.60:0/1505305538 shutdown_connections 2023-08-24T10:07:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.175+0000 7fcfec4ac640 1 -- 172.21.15.60:0/1505305538 wait complete. 2023-08-24T10:07:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.175+0000 7fcfec4ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.176+0000 7fcfec4ac640 1 Processor -- start 2023-08-24T10:07:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.176+0000 7fcfec4ac640 1 -- start start 2023-08-24T10:07:55.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.177+0000 7fcfec4ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfe4096860 0x7fcfe41c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:55.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.177+0000 7fcfec4ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe4098ce0 con 0x7fcfe4096860 2023-08-24T10:07:55.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.177+0000 7fcfea248640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfe4096860 0x7fcfe41c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:55.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.177+0000 7fcfea248640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfe4096860 0x7fcfe41c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35618/0 (socket says 172.21.15.60:35618) 2023-08-24T10:07:55.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.177+0000 7fcfea248640 1 -- 172.21.15.60:0/3143880247 learned_addr learned my addr 172.21.15.60:0/3143880247 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:55.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.177+0000 7fcfea248640 1 -- 172.21.15.60:0/3143880247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe412b2a0 con 0x7fcfe4096860 2023-08-24T10:07:55.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.177+0000 7fcfea248640 1 --2- 172.21.15.60:0/3143880247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfe4096860 0x7fcfe41c52a0 crc :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:55.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.177+0000 7fcfdaffd640 1 -- 172.21.15.60:0/3143880247 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcfe000c7e0 con 0x7fcfe4096860 2023-08-24T10:07:55.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.177+0000 7fcfdaffd640 1 -- 172.21.15.60:0/3143880247 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fcfe000c940 con 0x7fcfe4096860 2023-08-24T10:07:55.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.177+0000 7fcfec4ac640 1 -- 172.21.15.60:0/3143880247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe412b9e0 con 0x7fcfe4096860 2023-08-24T10:07:55.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.177+0000 7fcfdaffd640 1 -- 172.21.15.60:0/3143880247 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcfe000cbe0 con 0x7fcfe4096860 2023-08-24T10:07:55.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.177+0000 7fcfec4ac640 1 -- 172.21.15.60:0/3143880247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe41c4e70 con 0x7fcfe4096860 2023-08-24T10:07:55.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.178+0000 7fcfdaffd640 1 -- 172.21.15.60:0/3143880247 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fcfe0011310 con 0x7fcfe4096860 2023-08-24T10:07:55.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.178+0000 7fcfec4ac640 1 -- 172.21.15.60:0/3143880247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe4061800 con 0x7fcfe4096860 2023-08-24T10:07:55.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.178+0000 7fcfdaffd640 1 --2- 172.21.15.60:0/3143880247 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcfc8037130 0x7fcfc8039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:55.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.178+0000 7fcfdaffd640 1 -- 172.21.15.60:0/3143880247 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fcfe0046ad0 con 0x7fcfe4096860 2023-08-24T10:07:55.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.179+0000 7fcfe9a47640 1 --2- 172.21.15.60:0/3143880247 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcfc8037130 0x7fcfc8039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:55.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.179+0000 7fcfe9a47640 1 --2- 172.21.15.60:0/3143880247 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcfc8037130 0x7fcfc8039640 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:55.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.183+0000 7fcfdaffd640 1 -- 172.21.15.60:0/3143880247 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fcfe000ad00 con 0x7fcfe4096860 2023-08-24T10:07:55.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.357+0000 7fcfec4ac640 1 -- 172.21.15.60:0/3143880247 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcfe406d190 con 0x7fcfc8037130 2023-08-24T10:07:55.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.358+0000 7fcfdaffd640 1 -- 172.21.15.60:0/3143880247 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7fcfd4009df0 con 0x7fcfc8037130 2023-08-24T10:07:55.360 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:55.361 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:55.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.360+0000 7fcfec4ac640 1 -- 172.21.15.60:0/3143880247 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcfc8037130 msgr2=0x7fcfc8039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:55.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.360+0000 7fcfec4ac640 1 --2- 172.21.15.60:0/3143880247 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcfc8037130 0x7fcfc8039640 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:55.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.360+0000 7fcfec4ac640 1 -- 172.21.15.60:0/3143880247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfe4096860 msgr2=0x7fcfe41c52a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.360+0000 7fcfec4ac640 1 --2- 172.21.15.60:0/3143880247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfe4096860 0x7fcfe41c52a0 crc :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.360+0000 7fcfec4ac640 1 -- 172.21.15.60:0/3143880247 shutdown_connections 2023-08-24T10:07:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.360+0000 7fcfec4ac640 1 --2- 172.21.15.60:0/3143880247 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcfc8037130 0x7fcfc8039640 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:55.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.360+0000 7fcfec4ac640 1 --2- 172.21.15.60:0/3143880247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfe4096860 0x7fcfe41c52a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:55.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.360+0000 7fcfec4ac640 1 -- 172.21.15.60:0/3143880247 >> 172.21.15.60:0/3143880247 conn(0x7fcfe409e110 msgr2=0x7fcfe40950c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:55.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.360+0000 7fcfec4ac640 1 -- 172.21.15.60:0/3143880247 shutdown_connections 2023-08-24T10:07:55.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.360+0000 7fcfec4ac640 1 -- 172.21.15.60:0/3143880247 wait complete. 2023-08-24T10:07:55.372 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2023-08-24T10:07:53.529287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:55.373 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:55.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.479+0000 7f73aa8ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:55.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.483+0000 7f73aa8ec640 1 Processor -- start 2023-08-24T10:07:55.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.483+0000 7f73aa8ec640 1 -- start start 2023-08-24T10:07:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.483+0000 7f73aa8ec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f73a406d300 0x7f73a4097710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.483+0000 7f73aa8ec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a4099440 con 0x7f73a412bc80 2023-08-24T10:07:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.483+0000 7f73aa8ec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a409a050 con 0x7f73a406d300 2023-08-24T10:07:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.484+0000 7f73a3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f73a406d300 0x7f73a4097710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.484+0000 7f73a3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f73a406d300 0x7f73a4097710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35634/0 (socket says 172.21.15.60:35634) 2023-08-24T10:07:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.484+0000 7f73a3fff640 1 -- 172.21.15.60:0/2216152649 learned_addr learned my addr 172.21.15.60:0/2216152649 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.484+0000 7f73a37fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f73a412bc80 0x7f73a412ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54284/0 (socket says 172.21.15.60:54284) 2023-08-24T10:07:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.484+0000 7f73a3fff640 1 -- 172.21.15.60:0/2216152649 >> v1:172.21.15.60:6789/0 conn(0x7f73a412bc80 legacy=0x7f73a412ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.484+0000 7f73a3fff640 1 -- 172.21.15.60:0/2216152649 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a412b9d0 con 0x7f73a406d300 2023-08-24T10:07:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.484+0000 7f73a3fff640 1 --2- 172.21.15.60:0/2216152649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a406d300 0x7f73a4097710 crc :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cb7bac991cc462 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.484+0000 7f73a2ffd640 1 -- 172.21.15.60:0/2216152649 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7394009dc0 con 0x7f73a406d300 2023-08-24T10:07:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.484+0000 7f73a2ffd640 1 -- 172.21.15.60:0/2216152649 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f739400e230 con 0x7f73a406d300 2023-08-24T10:07:55.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.484+0000 7f73a2ffd640 1 -- 172.21.15.60:0/2216152649 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f739400e4d0 con 0x7f73a406d300 2023-08-24T10:07:55.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.485+0000 7f73aa8ec640 1 -- 172.21.15.60:0/2216152649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a406d300 msgr2=0x7f73a4097710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:55.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.485+0000 7f73aa8ec640 1 --2- 172.21.15.60:0/2216152649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a406d300 0x7f73a4097710 crc :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:55.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.485+0000 7f73aa8ec640 1 -- 172.21.15.60:0/2216152649 shutdown_connections 2023-08-24T10:07:55.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.485+0000 7f73aa8ec640 1 --2- 172.21.15.60:0/2216152649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a406d300 0x7f73a4097710 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.485+0000 7f73aa8ec640 1 -- 172.21.15.60:0/2216152649 >> 172.21.15.60:0/2216152649 conn(0x7f73a409e110 msgr2=0x7f73a409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.485+0000 7f73aa8ec640 1 -- 172.21.15.60:0/2216152649 shutdown_connections 2023-08-24T10:07:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.485+0000 7f73aa8ec640 1 -- 172.21.15.60:0/2216152649 wait complete. 2023-08-24T10:07:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.485+0000 7f73aa8ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.486+0000 7f73aa8ec640 1 Processor -- start 2023-08-24T10:07:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.486+0000 7f73aa8ec640 1 -- start start 2023-08-24T10:07:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.487+0000 7f73aa8ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a406d300 0x7f73a406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.487+0000 7f73aa8ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a40700b0 con 0x7f73a406d300 2023-08-24T10:07:55.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.487+0000 7f73a3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a406d300 0x7f73a406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:55.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.487+0000 7f73a3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a406d300 0x7f73a406e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35648/0 (socket says 172.21.15.60:35648) 2023-08-24T10:07:55.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.487+0000 7f73a3fff640 1 -- 172.21.15.60:0/2951663924 learned_addr learned my addr 172.21.15.60:0/2951663924 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:55.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.487+0000 7f73a3fff640 1 -- 172.21.15.60:0/2951663924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a406fce0 con 0x7f73a406d300 2023-08-24T10:07:55.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.487+0000 7f73a3fff640 1 --2- 172.21.15.60:0/2951663924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a406d300 0x7f73a406e0b0 crc :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:55.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.488+0000 7f73a0ff9640 1 -- 172.21.15.60:0/2951663924 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7394014540 con 0x7f73a406d300 2023-08-24T10:07:55.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.488+0000 7f73a0ff9640 1 -- 172.21.15.60:0/2951663924 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f73940146a0 con 0x7f73a406d300 2023-08-24T10:07:55.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.488+0000 7f73aa8ec640 1 -- 172.21.15.60:0/2951663924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a40891e0 con 0x7f73a406d300 2023-08-24T10:07:55.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.488+0000 7f73a0ff9640 1 -- 172.21.15.60:0/2951663924 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7394014940 con 0x7f73a406d300 2023-08-24T10:07:55.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.488+0000 7f73aa8ec640 1 -- 172.21.15.60:0/2951663924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a4070530 con 0x7f73a406d300 2023-08-24T10:07:55.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.488+0000 7f73a0ff9640 1 -- 172.21.15.60:0/2951663924 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f7394014af0 con 0x7f73a406d300 2023-08-24T10:07:55.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.488+0000 7f73aa8ec640 1 -- 172.21.15.60:0/2951663924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a4089820 con 0x7f73a406d300 2023-08-24T10:07:55.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.489+0000 7f73a0ff9640 1 --2- 172.21.15.60:0/2951663924 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7390037130 0x7f7390039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:55.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.489+0000 7f73a0ff9640 1 -- 172.21.15.60:0/2951663924 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f7394019040 con 0x7f73a406d300 2023-08-24T10:07:55.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.489+0000 7f73a37fe640 1 --2- 172.21.15.60:0/2951663924 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7390037130 0x7f7390039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:55.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.489+0000 7f73a37fe640 1 --2- 172.21.15.60:0/2951663924 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7390037130 0x7f7390039640 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:55.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.493+0000 7f73a0ff9640 1 -- 172.21.15.60:0/2951663924 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f73a4089820 con 0x7f73a406d300 2023-08-24T10:07:55.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.665+0000 7f73aa8ec640 1 -- 172.21.15.60:0/2951663924 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73a406d710 con 0x7f7390037130 2023-08-24T10:07:55.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.666+0000 7f73a0ff9640 1 -- 172.21.15.60:0/2951663924 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f73980070b0 con 0x7f7390037130 2023-08-24T10:07:55.668 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:55.669 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.668+0000 7f73aa8ec640 1 -- 172.21.15.60:0/2951663924 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7390037130 msgr2=0x7f7390039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:55.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.668+0000 7f73aa8ec640 1 --2- 172.21.15.60:0/2951663924 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7390037130 0x7f7390039640 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:55.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.668+0000 7f73aa8ec640 1 -- 172.21.15.60:0/2951663924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a406d300 msgr2=0x7f73a406e0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:55.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.668+0000 7f73aa8ec640 1 --2- 172.21.15.60:0/2951663924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a406d300 0x7f73a406e0b0 crc :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:55.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.668+0000 7f73aa8ec640 1 -- 172.21.15.60:0/2951663924 shutdown_connections 2023-08-24T10:07:55.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.668+0000 7f73aa8ec640 1 --2- 172.21.15.60:0/2951663924 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7390037130 0x7f7390039640 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:55.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.668+0000 7f73aa8ec640 1 --2- 172.21.15.60:0/2951663924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a406d300 0x7f73a406e0b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.668+0000 7f73aa8ec640 1 -- 172.21.15.60:0/2951663924 >> 172.21.15.60:0/2951663924 conn(0x7f73a409e110 msgr2=0x7f73a409afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.668+0000 7f73aa8ec640 1 -- 172.21.15.60:0/2951663924 shutdown_connections 2023-08-24T10:07:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.668+0000 7f73aa8ec640 1 -- 172.21.15.60:0/2951663924 wait complete. 2023-08-24T10:07:55.681 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2023-08-24T10:07:55.529567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:55.682 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:07:55.682 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:55.683 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:07:55.683 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:55.683 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:07:55.684 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:55.684 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:07:55.684 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:55.684 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:07:55.685 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:55.685 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:07:55.685 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:55.685 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:07:55.686 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:55.686 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:07:55.686 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:55.686 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:07:55.687 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:07:55.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:07:55.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.793+0000 7efcf095e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:55.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.796+0000 7efcf095e640 1 Processor -- start 2023-08-24T10:07:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.796+0000 7efcf095e640 1 -- start start 2023-08-24T10:07:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.797+0000 7efcf095e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efcec070c30 0x7efcec071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.797+0000 7efcf095e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcec06dfc0 con 0x7efcec0990b0 2023-08-24T10:07:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.797+0000 7efcf095e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcec06d9b0 con 0x7efcec070c30 2023-08-24T10:07:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.797+0000 7efcea59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efcec070c30 0x7efcec071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.797+0000 7efcea59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efcec070c30 0x7efcec071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35652/0 (socket says 172.21.15.60:35652) 2023-08-24T10:07:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.797+0000 7efcea59c640 1 -- 172.21.15.60:0/455508245 learned_addr learned my addr 172.21.15.60:0/455508245 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.797+0000 7efce9d9b640 1 --1- 172.21.15.60:0/455508245 >> v1:172.21.15.60:6789/0 conn(0x7efcec0990b0 0x7efcec0994c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54290/0 (socket says 172.21.15.60:54290) 2023-08-24T10:07:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.798+0000 7efcea59c640 1 -- 172.21.15.60:0/455508245 >> v1:172.21.15.60:6789/0 conn(0x7efcec0990b0 legacy=0x7efcec0994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.798+0000 7efcea59c640 1 -- 172.21.15.60:0/455508245 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcec06e6e0 con 0x7efcec070c30 2023-08-24T10:07:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.798+0000 7efcea59c640 1 --2- 172.21.15.60:0/455508245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcec070c30 0x7efcec071040 crc :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5609ddb5edf391 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.798+0000 7efce959a640 1 -- 172.21.15.60:0/455508245 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efcdc00e040 con 0x7efcec070c30 2023-08-24T10:07:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.798+0000 7efce959a640 1 -- 172.21.15.60:0/455508245 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7efcdc0091e0 con 0x7efcec070c30 2023-08-24T10:07:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.798+0000 7efce959a640 1 -- 172.21.15.60:0/455508245 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efcdc007730 con 0x7efcec070c30 2023-08-24T10:07:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.799+0000 7efcf095e640 1 -- 172.21.15.60:0/455508245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcec070c30 msgr2=0x7efcec071040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.799+0000 7efcf095e640 1 --2- 172.21.15.60:0/455508245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcec070c30 0x7efcec071040 crc :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.799+0000 7efcf095e640 1 -- 172.21.15.60:0/455508245 shutdown_connections 2023-08-24T10:07:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.799+0000 7efcf095e640 1 --2- 172.21.15.60:0/455508245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcec070c30 0x7efcec071040 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.799+0000 7efcf095e640 1 -- 172.21.15.60:0/455508245 >> 172.21.15.60:0/455508245 conn(0x7efcec09e110 msgr2=0x7efcec09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.799+0000 7efcf095e640 1 -- 172.21.15.60:0/455508245 shutdown_connections 2023-08-24T10:07:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.799+0000 7efcf095e640 1 -- 172.21.15.60:0/455508245 wait complete. 2023-08-24T10:07:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.799+0000 7efcf095e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.800+0000 7efcf095e640 1 Processor -- start 2023-08-24T10:07:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.800+0000 7efcf095e640 1 -- start start 2023-08-24T10:07:55.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.801+0000 7efcf095e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcec070c30 0x7efcec1c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:55.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.801+0000 7efcf095e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcec12dec0 con 0x7efcec070c30 2023-08-24T10:07:55.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.801+0000 7efcea59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcec070c30 0x7efcec1c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:55.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.801+0000 7efcea59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcec070c30 0x7efcec1c31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35662/0 (socket says 172.21.15.60:35662) 2023-08-24T10:07:55.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.801+0000 7efcea59c640 1 -- 172.21.15.60:0/865113682 learned_addr learned my addr 172.21.15.60:0/865113682 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:55.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.801+0000 7efcea59c640 1 -- 172.21.15.60:0/865113682 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcec12b9d0 con 0x7efcec070c30 2023-08-24T10:07:55.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.801+0000 7efcea59c640 1 --2- 172.21.15.60:0/865113682 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcec070c30 0x7efcec1c31e0 crc :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:55.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.801+0000 7efce2ffd640 1 -- 172.21.15.60:0/865113682 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efcdc00e040 con 0x7efcec070c30 2023-08-24T10:07:55.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.801+0000 7efce2ffd640 1 -- 172.21.15.60:0/865113682 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7efcdc012720 con 0x7efcec070c30 2023-08-24T10:07:55.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.801+0000 7efcf095e640 1 -- 172.21.15.60:0/865113682 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcec12ca70 con 0x7efcec070c30 2023-08-24T10:07:55.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.801+0000 7efce2ffd640 1 -- 172.21.15.60:0/865113682 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efcdc0129c0 con 0x7efcec070c30 2023-08-24T10:07:55.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.801+0000 7efcf095e640 1 -- 172.21.15.60:0/865113682 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcec12d210 con 0x7efcec070c30 2023-08-24T10:07:55.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.802+0000 7efce2ffd640 1 -- 172.21.15.60:0/865113682 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7efcdc011040 con 0x7efcec070c30 2023-08-24T10:07:55.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.802+0000 7efce2ffd640 1 --2- 172.21.15.60:0/865113682 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efccc037130 0x7efccc039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:55.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.802+0000 7efcf095e640 1 -- 172.21.15.60:0/865113682 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcec06e220 con 0x7efcec070c30 2023-08-24T10:07:55.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.802+0000 7efce2ffd640 1 -- 172.21.15.60:0/865113682 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7efcdc047a30 con 0x7efcec070c30 2023-08-24T10:07:55.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.803+0000 7efce9d9b640 1 --2- 172.21.15.60:0/865113682 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efccc037130 0x7efccc039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:55.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.803+0000 7efce9d9b640 1 --2- 172.21.15.60:0/865113682 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efccc037130 0x7efccc039640 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:55.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:55.807+0000 7efce2ffd640 1 -- 172.21.15.60:0/865113682 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7efcdc010ae0 con 0x7efcec070c30 2023-08-24T10:07:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.051+0000 7efcf095e640 1 -- 172.21.15.60:0/865113682 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7efcec061800 con 0x7efcec070c30 2023-08-24T10:07:56.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.052+0000 7efce2ffd640 1 -- 172.21.15.60:0/865113682 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7efcdc013e60 con 0x7efcec070c30 2023-08-24T10:07:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:56.055 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":186,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:07:56.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.054+0000 7efcf095e640 1 -- 172.21.15.60:0/865113682 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efccc037130 msgr2=0x7efccc039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:56.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.054+0000 7efcf095e640 1 --2- 172.21.15.60:0/865113682 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efccc037130 0x7efccc039640 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:56.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.054+0000 7efcf095e640 1 -- 172.21.15.60:0/865113682 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcec070c30 msgr2=0x7efcec1c31e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:56.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.054+0000 7efcf095e640 1 --2- 172.21.15.60:0/865113682 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcec070c30 0x7efcec1c31e0 crc :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:56.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.054+0000 7efcf095e640 1 -- 172.21.15.60:0/865113682 shutdown_connections 2023-08-24T10:07:56.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.054+0000 7efcf095e640 1 --2- 172.21.15.60:0/865113682 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efccc037130 0x7efccc039640 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.054+0000 7efcf095e640 1 --2- 172.21.15.60:0/865113682 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcec070c30 0x7efcec1c31e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.054+0000 7efcf095e640 1 -- 172.21.15.60:0/865113682 >> 172.21.15.60:0/865113682 conn(0x7efcec09e110 msgr2=0x7efcec09ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.054+0000 7efcf095e640 1 -- 172.21.15.60:0/865113682 shutdown_connections 2023-08-24T10:07:56.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.054+0000 7efcf095e640 1 -- 172.21.15.60:0/865113682 wait complete. 2023-08-24T10:07:56.066 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:07:56.067 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:56.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.172+0000 7f4bd8f1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:56.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.176+0000 7f4bd8f1e640 1 Processor -- start 2023-08-24T10:07:56.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.176+0000 7f4bd8f1e640 1 -- start start 2023-08-24T10:07:56.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.177+0000 7f4bd8f1e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4bd41337b0 0x7f4bd4135bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:56.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.177+0000 7f4bd8f1e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd412cfa0 con 0x7f4bd412d360 2023-08-24T10:07:56.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.177+0000 7f4bd8f1e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd412de80 con 0x7f4bd41337b0 2023-08-24T10:07:56.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.177+0000 7f4bd1d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4bd41337b0 0x7f4bd4135bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:56.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.177+0000 7f4bd1d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4bd41337b0 0x7f4bd4135bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35672/0 (socket says 172.21.15.60:35672) 2023-08-24T10:07:56.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.177+0000 7f4bd1d9b640 1 -- 172.21.15.60:0/3509997084 learned_addr learned my addr 172.21.15.60:0/3509997084 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:56.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.178+0000 7f4bd1d9b640 1 -- 172.21.15.60:0/3509997084 >> v1:172.21.15.60:6789/0 conn(0x7f4bd412d360 legacy=0x7f4bd412d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:56.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.178+0000 7f4bd1d9b640 1 -- 172.21.15.60:0/3509997084 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd4070d70 con 0x7f4bd41337b0 2023-08-24T10:07:56.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.178+0000 7f4bd1d9b640 1 --2- 172.21.15.60:0/3509997084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bd41337b0 0x7f4bd4135bd0 crc :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf901b577dd2bf1 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:56.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.178+0000 7f4bd159a640 1 -- 172.21.15.60:0/3509997084 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4bbc009dd0 con 0x7f4bd41337b0 2023-08-24T10:07:56.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.178+0000 7f4bd159a640 1 -- 172.21.15.60:0/3509997084 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4bbc00e230 con 0x7f4bd41337b0 2023-08-24T10:07:56.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.178+0000 7f4bd159a640 1 -- 172.21.15.60:0/3509997084 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4bbc00e4d0 con 0x7f4bd41337b0 2023-08-24T10:07:56.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.178+0000 7f4bd8f1e640 1 -- 172.21.15.60:0/3509997084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bd41337b0 msgr2=0x7f4bd4135bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:56.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.178+0000 7f4bd8f1e640 1 --2- 172.21.15.60:0/3509997084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bd41337b0 0x7f4bd4135bd0 crc :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:56.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.179+0000 7f4bd8f1e640 1 -- 172.21.15.60:0/3509997084 shutdown_connections 2023-08-24T10:07:56.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.179+0000 7f4bd8f1e640 1 --2- 172.21.15.60:0/3509997084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bd41337b0 0x7f4bd4135bd0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:56.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.179+0000 7f4bd8f1e640 1 -- 172.21.15.60:0/3509997084 >> 172.21.15.60:0/3509997084 conn(0x7f4bd409e610 msgr2=0x7f4bd409ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:56.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.179+0000 7f4bd8f1e640 1 -- 172.21.15.60:0/3509997084 shutdown_connections 2023-08-24T10:07:56.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.179+0000 7f4bd8f1e640 1 -- 172.21.15.60:0/3509997084 wait complete. 2023-08-24T10:07:56.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.179+0000 7f4bd8f1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:56.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.180+0000 7f4bd8f1e640 1 Processor -- start 2023-08-24T10:07:56.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.180+0000 7f4bd8f1e640 1 -- start start 2023-08-24T10:07:56.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.181+0000 7f4bd8f1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bd412d360 0x7f4bd4119e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:56.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.181+0000 7f4bd8f1e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd411c170 con 0x7f4bd412d360 2023-08-24T10:07:56.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.181+0000 7f4bd259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bd412d360 0x7f4bd4119e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:56.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.181+0000 7f4bd259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bd412d360 0x7f4bd4119e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35676/0 (socket says 172.21.15.60:35676) 2023-08-24T10:07:56.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.181+0000 7f4bd259c640 1 -- 172.21.15.60:0/483972541 learned_addr learned my addr 172.21.15.60:0/483972541 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:56.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.181+0000 7f4bd259c640 1 -- 172.21.15.60:0/483972541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd4119c10 con 0x7f4bd412d360 2023-08-24T10:07:56.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.181+0000 7f4bd259c640 1 --2- 172.21.15.60:0/483972541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bd412d360 0x7f4bd4119e30 crc :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:56.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.181+0000 7f4bcaffd640 1 -- 172.21.15.60:0/483972541 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4bcc00c7e0 con 0x7f4bd412d360 2023-08-24T10:07:56.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.181+0000 7f4bcaffd640 1 -- 172.21.15.60:0/483972541 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4bcc00c940 con 0x7f4bd412d360 2023-08-24T10:07:56.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.181+0000 7f4bd8f1e640 1 -- 172.21.15.60:0/483972541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd411b0d0 con 0x7f4bd412d360 2023-08-24T10:07:56.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.181+0000 7f4bcaffd640 1 -- 172.21.15.60:0/483972541 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4bcc00cbe0 con 0x7f4bd412d360 2023-08-24T10:07:56.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.181+0000 7f4bd8f1e640 1 -- 172.21.15.60:0/483972541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd411d5c0 con 0x7f4bd412d360 2023-08-24T10:07:56.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.182+0000 7f4bcaffd640 1 -- 172.21.15.60:0/483972541 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f4bcc011310 con 0x7f4bd412d360 2023-08-24T10:07:56.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.182+0000 7f4bcaffd640 1 --2- 172.21.15.60:0/483972541 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4bb4037130 0x7f4bb4039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:56.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.182+0000 7f4bcaffd640 1 -- 172.21.15.60:0/483972541 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f4bcc046ad0 con 0x7f4bd412d360 2023-08-24T10:07:56.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.182+0000 7f4bd8f1e640 1 -- 172.21.15.60:0/483972541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd41199d0 con 0x7f4bd412d360 2023-08-24T10:07:56.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.183+0000 7f4bd1d9b640 1 --2- 172.21.15.60:0/483972541 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4bb4037130 0x7f4bb4039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:56.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.183+0000 7f4bd1d9b640 1 --2- 172.21.15.60:0/483972541 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4bb4037130 0x7f4bb4039640 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:56.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.187+0000 7f4bcaffd640 1 -- 172.21.15.60:0/483972541 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f4bcc00ad00 con 0x7f4bd412d360 2023-08-24T10:07:56.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.359+0000 7f4bd8f1e640 1 -- 172.21.15.60:0/483972541 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4bd4070630 con 0x7f4bb4037130 2023-08-24T10:07:56.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.361+0000 7f4bcaffd640 1 -- 172.21.15.60:0/483972541 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f4bbc009da0 con 0x7f4bb4037130 2023-08-24T10:07:56.363 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:56.364 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:56.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.362+0000 7f4bd8f1e640 1 -- 172.21.15.60:0/483972541 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4bb4037130 msgr2=0x7f4bb4039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:56.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.362+0000 7f4bd8f1e640 1 --2- 172.21.15.60:0/483972541 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4bb4037130 0x7f4bb4039640 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:56.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.362+0000 7f4bd8f1e640 1 -- 172.21.15.60:0/483972541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bd412d360 msgr2=0x7f4bd4119e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:56.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.362+0000 7f4bd8f1e640 1 --2- 172.21.15.60:0/483972541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bd412d360 0x7f4bd4119e30 crc :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:56.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.362+0000 7f4bd8f1e640 1 -- 172.21.15.60:0/483972541 shutdown_connections 2023-08-24T10:07:56.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.362+0000 7f4bd8f1e640 1 --2- 172.21.15.60:0/483972541 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4bb4037130 0x7f4bb4039640 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:56.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.362+0000 7f4bd8f1e640 1 --2- 172.21.15.60:0/483972541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bd412d360 0x7f4bd4119e30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:56.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.362+0000 7f4bd8f1e640 1 -- 172.21.15.60:0/483972541 >> 172.21.15.60:0/483972541 conn(0x7f4bd409e610 msgr2=0x7f4bd406f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:56.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.362+0000 7f4bd8f1e640 1 -- 172.21.15.60:0/483972541 shutdown_connections 2023-08-24T10:07:56.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:56.362+0000 7f4bd8f1e640 1 -- 172.21.15.60:0/483972541 wait complete. 2023-08-24T10:07:56.377 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2023-08-24T10:07:55.529567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:59.176 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~172s 2023-08-24T10:07:59.177 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~172s 2023-08-24T10:07:59.177 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~172s 2023-08-24T10:07:59.177 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~172s 2023-08-24T10:07:59.379 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:59.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.499+0000 7f720df1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:59.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.503+0000 7f720df1e640 1 Processor -- start 2023-08-24T10:07:59.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.503+0000 7f720df1e640 1 -- start start 2023-08-24T10:07:59.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.504+0000 7f720df1e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7208096860 0x7f7208096c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:59.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.504+0000 7f720df1e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72080971b0 con 0x7f72080990b0 2023-08-24T10:07:59.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.504+0000 7f720df1e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f720806db70 con 0x7f7208096860 2023-08-24T10:07:59.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.504+0000 7f72077fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7208096860 0x7f7208096c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:59.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.504+0000 7f72077fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7208096860 0x7f7208096c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44194/0 (socket says 172.21.15.60:44194) 2023-08-24T10:07:59.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.504+0000 7f72077fe640 1 -- 172.21.15.60:0/3178387550 learned_addr learned my addr 172.21.15.60:0/3178387550 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:59.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.504+0000 7f7206ffd640 1 --1- 172.21.15.60:0/3178387550 >> v1:172.21.15.60:6789/0 conn(0x7f72080990b0 0x7f72080994c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46880/0 (socket says 172.21.15.60:46880) 2023-08-24T10:07:59.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.504+0000 7f72077fe640 1 -- 172.21.15.60:0/3178387550 >> v1:172.21.15.60:6789/0 conn(0x7f72080990b0 legacy=0x7f72080994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:59.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.504+0000 7f72077fe640 1 -- 172.21.15.60:0/3178387550 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720806e320 con 0x7f7208096860 2023-08-24T10:07:59.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.505+0000 7f72077fe640 1 --2- 172.21.15.60:0/3178387550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7208096860 0x7f7208096c70 crc :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8eed5b50c9dfa0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:59.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.505+0000 7f72067fc640 1 -- 172.21.15.60:0/3178387550 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f71f8009dd0 con 0x7f7208096860 2023-08-24T10:07:59.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.505+0000 7f72067fc640 1 -- 172.21.15.60:0/3178387550 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f71f800e230 con 0x7f7208096860 2023-08-24T10:07:59.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.505+0000 7f72067fc640 1 -- 172.21.15.60:0/3178387550 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f71f800e4d0 con 0x7f7208096860 2023-08-24T10:07:59.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.505+0000 7f720df1e640 1 -- 172.21.15.60:0/3178387550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7208096860 msgr2=0x7f7208096c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:59.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.505+0000 7f720df1e640 1 --2- 172.21.15.60:0/3178387550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7208096860 0x7f7208096c70 crc :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:59.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.505+0000 7f720df1e640 1 -- 172.21.15.60:0/3178387550 shutdown_connections 2023-08-24T10:07:59.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.505+0000 7f720df1e640 1 --2- 172.21.15.60:0/3178387550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7208096860 0x7f7208096c70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:59.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.505+0000 7f720df1e640 1 -- 172.21.15.60:0/3178387550 >> 172.21.15.60:0/3178387550 conn(0x7f720809e110 msgr2=0x7f720809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:59.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.505+0000 7f720df1e640 1 -- 172.21.15.60:0/3178387550 shutdown_connections 2023-08-24T10:07:59.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.505+0000 7f720df1e640 1 -- 172.21.15.60:0/3178387550 wait complete. 2023-08-24T10:07:59.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.506+0000 7f720df1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:59.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.507+0000 7f720df1e640 1 Processor -- start 2023-08-24T10:07:59.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.507+0000 7f720df1e640 1 -- start start 2023-08-24T10:07:59.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.507+0000 7f720df1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7208096860 0x7f720812af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:59.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.507+0000 7f720df1e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7208129890 con 0x7f7208096860 2023-08-24T10:07:59.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.508+0000 7f72077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7208096860 0x7f720812af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:59.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.508+0000 7f72077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7208096860 0x7f720812af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44204/0 (socket says 172.21.15.60:44204) 2023-08-24T10:07:59.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.508+0000 7f72077fe640 1 -- 172.21.15.60:0/3626556848 learned_addr learned my addr 172.21.15.60:0/3626556848 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:59.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.508+0000 7f72077fe640 1 -- 172.21.15.60:0/3626556848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72081bf650 con 0x7f7208096860 2023-08-24T10:07:59.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.508+0000 7f72077fe640 1 --2- 172.21.15.60:0/3626556848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7208096860 0x7f720812af10 crc :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:59.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.508+0000 7f71effff640 1 -- 172.21.15.60:0/3626556848 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f71f8014410 con 0x7f7208096860 2023-08-24T10:07:59.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.508+0000 7f71effff640 1 -- 172.21.15.60:0/3626556848 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f71f8014570 con 0x7f7208096860 2023-08-24T10:07:59.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.508+0000 7f720df1e640 1 -- 172.21.15.60:0/3626556848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72081bf960 con 0x7f7208096860 2023-08-24T10:07:59.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.508+0000 7f71effff640 1 -- 172.21.15.60:0/3626556848 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f71f8014810 con 0x7f7208096860 2023-08-24T10:07:59.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.508+0000 7f720df1e640 1 -- 172.21.15.60:0/3626556848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720812a940 con 0x7f7208096860 2023-08-24T10:07:59.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.509+0000 7f71effff640 1 -- 172.21.15.60:0/3626556848 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f71f80149c0 con 0x7f7208096860 2023-08-24T10:07:59.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.509+0000 7f720df1e640 1 -- 172.21.15.60:0/3626556848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7208061800 con 0x7f7208096860 2023-08-24T10:07:59.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.509+0000 7f71effff640 1 --2- 172.21.15.60:0/3626556848 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f71f4037130 0x7f71f4039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:59.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.509+0000 7f71effff640 1 -- 172.21.15.60:0/3626556848 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f71f8019040 con 0x7f7208096860 2023-08-24T10:07:59.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.509+0000 7f7206ffd640 1 --2- 172.21.15.60:0/3626556848 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f71f4037130 0x7f71f4039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:59.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.510+0000 7f7206ffd640 1 --2- 172.21.15.60:0/3626556848 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f71f4037130 0x7f71f4039640 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:59.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.514+0000 7f71effff640 1 -- 172.21.15.60:0/3626556848 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f71f800f8f0 con 0x7f7208096860 2023-08-24T10:07:59.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.685+0000 7f720df1e640 1 -- 172.21.15.60:0/3626556848 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f720812bf20 con 0x7f71f4037130 2023-08-24T10:07:59.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.686+0000 7f71effff640 1 -- 172.21.15.60:0/3626556848 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f720812bf20 con 0x7f71f4037130 2023-08-24T10:07:59.689 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:59.689 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:59.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.688+0000 7f720df1e640 1 -- 172.21.15.60:0/3626556848 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f71f4037130 msgr2=0x7f71f4039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:59.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.688+0000 7f720df1e640 1 --2- 172.21.15.60:0/3626556848 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f71f4037130 0x7f71f4039640 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.688+0000 7f720df1e640 1 -- 172.21.15.60:0/3626556848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7208096860 msgr2=0x7f720812af10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.688+0000 7f720df1e640 1 --2- 172.21.15.60:0/3626556848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7208096860 0x7f720812af10 crc :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.688+0000 7f720df1e640 1 -- 172.21.15.60:0/3626556848 shutdown_connections 2023-08-24T10:07:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.688+0000 7f720df1e640 1 --2- 172.21.15.60:0/3626556848 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f71f4037130 0x7f71f4039640 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:59.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.688+0000 7f720df1e640 1 --2- 172.21.15.60:0/3626556848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7208096860 0x7f720812af10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:59.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.688+0000 7f720df1e640 1 -- 172.21.15.60:0/3626556848 >> 172.21.15.60:0/3626556848 conn(0x7f720809e110 msgr2=0x7f72081b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:59.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.688+0000 7f720df1e640 1 -- 172.21.15.60:0/3626556848 shutdown_connections 2023-08-24T10:07:59.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.688+0000 7f720df1e640 1 -- 172.21.15.60:0/3626556848 wait complete. 2023-08-24T10:07:59.700 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2023-08-24T10:07:59.530122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:07:59.702 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:07:59.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.807+0000 7f30cdd9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:59.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.811+0000 7f30cdd9e640 1 Processor -- start 2023-08-24T10:07:59.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.811+0000 7f30cdd9e640 1 -- start start 2023-08-24T10:07:59.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.812+0000 7f30cdd9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f30c8070110 0x7f30c81356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:59.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.812+0000 7f30cdd9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c8096f10 con 0x7f30c8071e40 2023-08-24T10:07:59.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.812+0000 7f30cdd9e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c80977f0 con 0x7f30c8070110 2023-08-24T10:07:59.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.812+0000 7f30c6ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f30c8070110 0x7f30c81356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:59.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.812+0000 7f30c6ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f30c8070110 0x7f30c81356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44216/0 (socket says 172.21.15.60:44216) 2023-08-24T10:07:59.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.812+0000 7f30c6ffd640 1 -- 172.21.15.60:0/3910809296 learned_addr learned my addr 172.21.15.60:0/3910809296 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:59.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.812+0000 7f30c6ffd640 1 -- 172.21.15.60:0/3910809296 >> v1:172.21.15.60:6789/0 conn(0x7f30c8071e40 legacy=0x7f30c8096800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:59.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.812+0000 7f30c6ffd640 1 -- 172.21.15.60:0/3910809296 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c809a280 con 0x7f30c8070110 2023-08-24T10:07:59.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.813+0000 7f30c6ffd640 1 --2- 172.21.15.60:0/3910809296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c8070110 0x7f30c81356a0 crc :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7d55dd4b2b875a server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:59.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.813+0000 7f30c67fc640 1 -- 172.21.15.60:0/3910809296 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f30bc009dd0 con 0x7f30c8070110 2023-08-24T10:07:59.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.813+0000 7f30c67fc640 1 -- 172.21.15.60:0/3910809296 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f30bc00e230 con 0x7f30c8070110 2023-08-24T10:07:59.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.813+0000 7f30c67fc640 1 -- 172.21.15.60:0/3910809296 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f30bc00e4d0 con 0x7f30c8070110 2023-08-24T10:07:59.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.813+0000 7f30cdd9e640 1 -- 172.21.15.60:0/3910809296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c8070110 msgr2=0x7f30c81356a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:59.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.813+0000 7f30cdd9e640 1 --2- 172.21.15.60:0/3910809296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c8070110 0x7f30c81356a0 crc :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:59.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.813+0000 7f30cdd9e640 1 -- 172.21.15.60:0/3910809296 shutdown_connections 2023-08-24T10:07:59.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.813+0000 7f30cdd9e640 1 --2- 172.21.15.60:0/3910809296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c8070110 0x7f30c81356a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:07:59.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.813+0000 7f30cdd9e640 1 -- 172.21.15.60:0/3910809296 >> 172.21.15.60:0/3910809296 conn(0x7f30c809e110 msgr2=0x7f30c809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:07:59.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.813+0000 7f30cdd9e640 1 -- 172.21.15.60:0/3910809296 shutdown_connections 2023-08-24T10:07:59.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.813+0000 7f30cdd9e640 1 -- 172.21.15.60:0/3910809296 wait complete. 2023-08-24T10:07:59.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.814+0000 7f30cdd9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:07:59.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.815+0000 7f30cdd9e640 1 Processor -- start 2023-08-24T10:07:59.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.815+0000 7f30cdd9e640 1 -- start start 2023-08-24T10:07:59.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.815+0000 7f30cdd9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c8070110 0x7f30c8093460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:59.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.815+0000 7f30cdd9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c808ff10 con 0x7f30c8070110 2023-08-24T10:07:59.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.815+0000 7f30c77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c8070110 0x7f30c8093460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:59.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.816+0000 7f30c77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c8070110 0x7f30c8093460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44232/0 (socket says 172.21.15.60:44232) 2023-08-24T10:07:59.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.816+0000 7f30c77fe640 1 -- 172.21.15.60:0/1842914244 learned_addr learned my addr 172.21.15.60:0/1842914244 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:07:59.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.816+0000 7f30c77fe640 1 -- 172.21.15.60:0/1842914244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c8090680 con 0x7f30c8070110 2023-08-24T10:07:59.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.816+0000 7f30c77fe640 1 --2- 172.21.15.60:0/1842914244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c8070110 0x7f30c8093460 crc :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:59.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.816+0000 7f30affff640 1 -- 172.21.15.60:0/1842914244 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f30b800c860 con 0x7f30c8070110 2023-08-24T10:07:59.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.816+0000 7f30cdd9e640 1 -- 172.21.15.60:0/1842914244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c8092e80 con 0x7f30c8070110 2023-08-24T10:07:59.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.816+0000 7f30affff640 1 -- 172.21.15.60:0/1842914244 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f30b800c9c0 con 0x7f30c8070110 2023-08-24T10:07:59.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.816+0000 7f30affff640 1 -- 172.21.15.60:0/1842914244 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f30b800cc60 con 0x7f30c8070110 2023-08-24T10:07:59.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.816+0000 7f30cdd9e640 1 -- 172.21.15.60:0/1842914244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c8092b70 con 0x7f30c8070110 2023-08-24T10:07:59.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.817+0000 7f30affff640 1 -- 172.21.15.60:0/1842914244 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f30b8011310 con 0x7f30c8070110 2023-08-24T10:07:59.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.817+0000 7f30cdd9e640 1 -- 172.21.15.60:0/1842914244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c8061800 con 0x7f30c8070110 2023-08-24T10:07:59.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.817+0000 7f30affff640 1 --2- 172.21.15.60:0/1842914244 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f30b4037130 0x7f30b4039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:07:59.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.817+0000 7f30affff640 1 -- 172.21.15.60:0/1842914244 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f30b8046a90 con 0x7f30c8070110 2023-08-24T10:07:59.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.817+0000 7f30c6ffd640 1 --2- 172.21.15.60:0/1842914244 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f30b4037130 0x7f30b4039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:07:59.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.818+0000 7f30c6ffd640 1 --2- 172.21.15.60:0/1842914244 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f30b4037130 0x7f30b4039640 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:07:59.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.822+0000 7f30affff640 1 -- 172.21.15.60:0/1842914244 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f30b8011e60 con 0x7f30c8070110 2023-08-24T10:07:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.992+0000 7f30cdd9e640 1 -- 172.21.15.60:0/1842914244 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f30c80932f0 con 0x7f30b4037130 2023-08-24T10:07:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.994+0000 7f30affff640 1 -- 172.21.15.60:0/1842914244 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f30c80932f0 con 0x7f30b4037130 2023-08-24T10:07:59.996 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:07:59.997 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:07:59.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.995+0000 7f30cdd9e640 1 -- 172.21.15.60:0/1842914244 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f30b4037130 msgr2=0x7f30b4039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:07:59.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.995+0000 7f30cdd9e640 1 --2- 172.21.15.60:0/1842914244 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f30b4037130 0x7f30b4039640 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:00.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.995+0000 7f30cdd9e640 1 -- 172.21.15.60:0/1842914244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c8070110 msgr2=0x7f30c8093460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:00.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.995+0000 7f30cdd9e640 1 --2- 172.21.15.60:0/1842914244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c8070110 0x7f30c8093460 crc :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:00.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.995+0000 7f30cdd9e640 1 -- 172.21.15.60:0/1842914244 shutdown_connections 2023-08-24T10:08:00.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.995+0000 7f30cdd9e640 1 --2- 172.21.15.60:0/1842914244 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f30b4037130 0x7f30b4039640 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:00.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.995+0000 7f30cdd9e640 1 --2- 172.21.15.60:0/1842914244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c8070110 0x7f30c8093460 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:00.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.995+0000 7f30cdd9e640 1 -- 172.21.15.60:0/1842914244 >> 172.21.15.60:0/1842914244 conn(0x7f30c809e110 msgr2=0x7f30c8098490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:00.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.995+0000 7f30cdd9e640 1 -- 172.21.15.60:0/1842914244 shutdown_connections 2023-08-24T10:08:00.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:07:59.995+0000 7f30cdd9e640 1 -- 172.21.15.60:0/1842914244 wait complete. 2023-08-24T10:08:00.008 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2023-08-24T10:07:59.530122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:00.009 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:08:00.010 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:00.010 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:08:00.011 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:00.011 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:08:00.012 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:00.012 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:08:00.012 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:00.013 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:08:00.013 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:00.014 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:08:00.015 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:00.015 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:08:00.016 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:00.016 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:08:00.016 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:00.017 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:08:00.017 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:00.018 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:08:00.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.124+0000 7f39bc463640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:00.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.127+0000 7f39bc463640 1 Processor -- start 2023-08-24T10:08:00.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.127+0000 7f39bc463640 1 -- start start 2023-08-24T10:08:00.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.128+0000 7f39bc463640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39b4131d20 0x7f39b4134140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:00.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.128+0000 7f39bc463640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b409a650 con 0x7f39b412ece0 2023-08-24T10:08:00.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.128+0000 7f39bc463640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b406da00 con 0x7f39b4131d20 2023-08-24T10:08:00.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.128+0000 7f39b99fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39b4131d20 0x7f39b4134140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:00.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.128+0000 7f39b99fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39b4131d20 0x7f39b4134140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44240/0 (socket says 172.21.15.60:44240) 2023-08-24T10:08:00.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.128+0000 7f39b99fe640 1 -- 172.21.15.60:0/2244944458 learned_addr learned my addr 172.21.15.60:0/2244944458 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:00.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.128+0000 7f39ba1ff640 1 --1- 172.21.15.60:0/2244944458 >> v1:172.21.15.60:6789/0 conn(0x7f39b412ece0 0x7f39b412f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46904/0 (socket says 172.21.15.60:46904) 2023-08-24T10:08:00.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.129+0000 7f39b99fe640 1 -- 172.21.15.60:0/2244944458 >> v1:172.21.15.60:6789/0 conn(0x7f39b412ece0 legacy=0x7f39b412f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:00.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.129+0000 7f39b99fe640 1 -- 172.21.15.60:0/2244944458 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b412ea10 con 0x7f39b4131d20 2023-08-24T10:08:00.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.129+0000 7f39b99fe640 1 --2- 172.21.15.60:0/2244944458 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b4131d20 0x7f39b4134140 crc :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0926b5dd7479a04 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:00.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.129+0000 7f39b91fd640 1 -- 172.21.15.60:0/2244944458 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f39a4007bd0 con 0x7f39b4131d20 2023-08-24T10:08:00.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.129+0000 7f39b91fd640 1 -- 172.21.15.60:0/2244944458 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f39a400a040 con 0x7f39b4131d20 2023-08-24T10:08:00.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.129+0000 7f39b91fd640 1 -- 172.21.15.60:0/2244944458 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f39a4007e60 con 0x7f39b4131d20 2023-08-24T10:08:00.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.130+0000 7f39bc463640 1 -- 172.21.15.60:0/2244944458 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b4131d20 msgr2=0x7f39b4134140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:00.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.130+0000 7f39bc463640 1 --2- 172.21.15.60:0/2244944458 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b4131d20 0x7f39b4134140 crc :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:00.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.130+0000 7f39bc463640 1 -- 172.21.15.60:0/2244944458 shutdown_connections 2023-08-24T10:08:00.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.130+0000 7f39bc463640 1 --2- 172.21.15.60:0/2244944458 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b4131d20 0x7f39b4134140 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:00.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.130+0000 7f39bc463640 1 -- 172.21.15.60:0/2244944458 >> 172.21.15.60:0/2244944458 conn(0x7f39b409e110 msgr2=0x7f39b409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:00.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.130+0000 7f39bc463640 1 -- 172.21.15.60:0/2244944458 shutdown_connections 2023-08-24T10:08:00.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.130+0000 7f39bc463640 1 -- 172.21.15.60:0/2244944458 wait complete. 2023-08-24T10:08:00.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.130+0000 7f39bc463640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:00.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.131+0000 7f39bc463640 1 Processor -- start 2023-08-24T10:08:00.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.131+0000 7f39bc463640 1 -- start start 2023-08-24T10:08:00.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.132+0000 7f39bc463640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b412ece0 0x7f39b4070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:00.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.132+0000 7f39bc463640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b4071b00 con 0x7f39b412ece0 2023-08-24T10:08:00.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.132+0000 7f39ba1ff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b412ece0 0x7f39b4070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:00.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.132+0000 7f39ba1ff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b412ece0 0x7f39b4070090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44250/0 (socket says 172.21.15.60:44250) 2023-08-24T10:08:00.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.132+0000 7f39ba1ff640 1 -- 172.21.15.60:0/1972613480 learned_addr learned my addr 172.21.15.60:0/1972613480 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:00.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.132+0000 7f39ba1ff640 1 -- 172.21.15.60:0/1972613480 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b4127400 con 0x7f39b412ece0 2023-08-24T10:08:00.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.132+0000 7f39ba1ff640 1 --2- 172.21.15.60:0/1972613480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b412ece0 0x7f39b4070090 crc :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:00.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.132+0000 7f39aaffd640 1 -- 172.21.15.60:0/1972613480 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f39b000ea70 con 0x7f39b412ece0 2023-08-24T10:08:00.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.132+0000 7f39aaffd640 1 -- 172.21.15.60:0/1972613480 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f39b000ebd0 con 0x7f39b412ece0 2023-08-24T10:08:00.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.132+0000 7f39bc463640 1 -- 172.21.15.60:0/1972613480 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b4126860 con 0x7f39b412ece0 2023-08-24T10:08:00.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.132+0000 7f39aaffd640 1 -- 172.21.15.60:0/1972613480 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f39b000ee70 con 0x7f39b412ece0 2023-08-24T10:08:00.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.132+0000 7f39bc463640 1 -- 172.21.15.60:0/1972613480 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b4125780 con 0x7f39b412ece0 2023-08-24T10:08:00.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.133+0000 7f39aaffd640 1 -- 172.21.15.60:0/1972613480 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f39b00079f0 con 0x7f39b412ece0 2023-08-24T10:08:00.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.133+0000 7f39bc463640 1 -- 172.21.15.60:0/1972613480 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b4061800 con 0x7f39b412ece0 2023-08-24T10:08:00.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.133+0000 7f39aaffd640 1 --2- 172.21.15.60:0/1972613480 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3998037130 0x7f3998039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:00.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.133+0000 7f39aaffd640 1 -- 172.21.15.60:0/1972613480 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f39b0048610 con 0x7f39b412ece0 2023-08-24T10:08:00.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.134+0000 7f39b99fe640 1 --2- 172.21.15.60:0/1972613480 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3998037130 0x7f3998039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:00.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.134+0000 7f39b99fe640 1 --2- 172.21.15.60:0/1972613480 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3998037130 0x7f3998039640 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:00.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.138+0000 7f39aaffd640 1 -- 172.21.15.60:0/1972613480 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f39b000fb80 con 0x7f39b412ece0 2023-08-24T10:08:00.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.381+0000 7f39bc463640 1 -- 172.21.15.60:0/1972613480 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f39b4073730 con 0x7f39b412ece0 2023-08-24T10:08:00.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.382+0000 7f39aaffd640 1 -- 172.21.15.60:0/1972613480 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f39b000f150 con 0x7f39b412ece0 2023-08-24T10:08:00.384 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:00.384 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":191,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:08:00.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.383+0000 7f39bc463640 1 -- 172.21.15.60:0/1972613480 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3998037130 msgr2=0x7f3998039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:00.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.383+0000 7f39bc463640 1 --2- 172.21.15.60:0/1972613480 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3998037130 0x7f3998039640 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:00.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.383+0000 7f39bc463640 1 -- 172.21.15.60:0/1972613480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b412ece0 msgr2=0x7f39b4070090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:00.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.383+0000 7f39bc463640 1 --2- 172.21.15.60:0/1972613480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b412ece0 0x7f39b4070090 crc :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:00.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.383+0000 7f39bc463640 1 -- 172.21.15.60:0/1972613480 shutdown_connections 2023-08-24T10:08:00.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.383+0000 7f39bc463640 1 --2- 172.21.15.60:0/1972613480 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3998037130 0x7f3998039640 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:00.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.383+0000 7f39bc463640 1 --2- 172.21.15.60:0/1972613480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39b412ece0 0x7f39b4070090 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:00.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.383+0000 7f39bc463640 1 -- 172.21.15.60:0/1972613480 >> 172.21.15.60:0/1972613480 conn(0x7f39b409e110 msgr2=0x7f39b4096820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:00.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.383+0000 7f39bc463640 1 -- 172.21.15.60:0/1972613480 shutdown_connections 2023-08-24T10:08:00.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.383+0000 7f39bc463640 1 -- 172.21.15.60:0/1972613480 wait complete. 2023-08-24T10:08:00.396 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:08:00.397 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:00.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.502+0000 7f038931e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:00.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.506+0000 7f038931e640 1 Processor -- start 2023-08-24T10:08:00.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.506+0000 7f038931e640 1 -- start start 2023-08-24T10:08:00.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.507+0000 7f038931e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f038406d300 0x7f038412bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:00.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.507+0000 7f038931e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0384099440 con 0x7f038412c1c0 2023-08-24T10:08:00.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.507+0000 7f038931e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f038409a050 con 0x7f038406d300 2023-08-24T10:08:00.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.507+0000 7f0382ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f038406d300 0x7f038412bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:00.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.507+0000 7f0382ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f038406d300 0x7f038412bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44266/0 (socket says 172.21.15.60:44266) 2023-08-24T10:08:00.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.507+0000 7f0382ffd640 1 -- 172.21.15.60:0/2354626177 learned_addr learned my addr 172.21.15.60:0/2354626177 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:00.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.507+0000 7f037bfff640 1 --1- 172.21.15.60:0/2354626177 >> v1:172.21.15.60:6789/0 conn(0x7f038412c1c0 0x7f038412ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46918/0 (socket says 172.21.15.60:46918) 2023-08-24T10:08:00.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.507+0000 7f0382ffd640 1 -- 172.21.15.60:0/2354626177 >> v1:172.21.15.60:6789/0 conn(0x7f038412c1c0 legacy=0x7f038412ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:00.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.507+0000 7f0382ffd640 1 -- 172.21.15.60:0/2354626177 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038412b9d0 con 0x7f038406d300 2023-08-24T10:08:00.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.508+0000 7f0382ffd640 1 --2- 172.21.15.60:0/2354626177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038406d300 0x7f038412bc80 crc :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18475fc58aa41a25 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:00.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.508+0000 7f03827fc640 1 -- 172.21.15.60:0/2354626177 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0374009dd0 con 0x7f038406d300 2023-08-24T10:08:00.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.508+0000 7f03827fc640 1 -- 172.21.15.60:0/2354626177 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f037400e230 con 0x7f038406d300 2023-08-24T10:08:00.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.508+0000 7f03827fc640 1 -- 172.21.15.60:0/2354626177 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f037400e4d0 con 0x7f038406d300 2023-08-24T10:08:00.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.508+0000 7f038931e640 1 -- 172.21.15.60:0/2354626177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038406d300 msgr2=0x7f038412bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:00.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.508+0000 7f038931e640 1 --2- 172.21.15.60:0/2354626177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038406d300 0x7f038412bc80 crc :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:00.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.508+0000 7f038931e640 1 -- 172.21.15.60:0/2354626177 shutdown_connections 2023-08-24T10:08:00.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.508+0000 7f038931e640 1 --2- 172.21.15.60:0/2354626177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038406d300 0x7f038412bc80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:00.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.509+0000 7f038931e640 1 -- 172.21.15.60:0/2354626177 >> 172.21.15.60:0/2354626177 conn(0x7f038409e110 msgr2=0x7f038409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:00.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.509+0000 7f038931e640 1 -- 172.21.15.60:0/2354626177 shutdown_connections 2023-08-24T10:08:00.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.509+0000 7f038931e640 1 -- 172.21.15.60:0/2354626177 wait complete. 2023-08-24T10:08:00.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.509+0000 7f038931e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:00.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.510+0000 7f038931e640 1 Processor -- start 2023-08-24T10:08:00.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.510+0000 7f038931e640 1 -- start start 2023-08-24T10:08:00.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.510+0000 7f038931e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038406d300 0x7f038406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:00.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.510+0000 7f038931e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0384089370 con 0x7f038406d300 2023-08-24T10:08:00.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.510+0000 7f0382ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038406d300 0x7f038406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:00.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.510+0000 7f0382ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038406d300 0x7f038406e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44274/0 (socket says 172.21.15.60:44274) 2023-08-24T10:08:00.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.510+0000 7f0382ffd640 1 -- 172.21.15.60:0/1689778663 learned_addr learned my addr 172.21.15.60:0/1689778663 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:00.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.511+0000 7f0382ffd640 1 -- 172.21.15.60:0/1689778663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0384096f10 con 0x7f038406d300 2023-08-24T10:08:00.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.511+0000 7f0382ffd640 1 --2- 172.21.15.60:0/1689778663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038406d300 0x7f038406e9f0 crc :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:00.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.511+0000 7f037b7fe640 1 -- 172.21.15.60:0/1689778663 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0374014410 con 0x7f038406d300 2023-08-24T10:08:00.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.511+0000 7f037b7fe640 1 -- 172.21.15.60:0/1689778663 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0374014570 con 0x7f038406d300 2023-08-24T10:08:00.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.511+0000 7f038931e640 1 -- 172.21.15.60:0/1689778663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0384089560 con 0x7f038406d300 2023-08-24T10:08:00.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.511+0000 7f037b7fe640 1 -- 172.21.15.60:0/1689778663 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0374014810 con 0x7f038406d300 2023-08-24T10:08:00.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.511+0000 7f038931e640 1 -- 172.21.15.60:0/1689778663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0384088ff0 con 0x7f038406d300 2023-08-24T10:08:00.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.512+0000 7f037b7fe640 1 -- 172.21.15.60:0/1689778663 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f03740149c0 con 0x7f038406d300 2023-08-24T10:08:00.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.512+0000 7f037b7fe640 1 --2- 172.21.15.60:0/1689778663 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0364037130 0x7f0364039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:00.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.512+0000 7f037b7fe640 1 -- 172.21.15.60:0/1689778663 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f0374019040 con 0x7f038406d300 2023-08-24T10:08:00.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.512+0000 7f038931e640 1 -- 172.21.15.60:0/1689778663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f038406e840 con 0x7f038406d300 2023-08-24T10:08:00.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.512+0000 7f037bfff640 1 --2- 172.21.15.60:0/1689778663 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0364037130 0x7f0364039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:00.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.513+0000 7f037bfff640 1 --2- 172.21.15.60:0/1689778663 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0364037130 0x7f0364039640 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:00.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.517+0000 7f037b7fe640 1 -- 172.21.15.60:0/1689778663 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f037400fb20 con 0x7f038406d300 2023-08-24T10:08:00.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.689+0000 7f038931e640 1 -- 172.21.15.60:0/1689778663 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f038406da20 con 0x7f0364037130 2023-08-24T10:08:00.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.690+0000 7f037b7fe640 1 -- 172.21.15.60:0/1689778663 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f036c0070b0 con 0x7f0364037130 2023-08-24T10:08:00.693 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:00.696 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:00.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.692+0000 7f038931e640 1 -- 172.21.15.60:0/1689778663 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0364037130 msgr2=0x7f0364039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:00.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.692+0000 7f038931e640 1 --2- 172.21.15.60:0/1689778663 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0364037130 0x7f0364039640 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:00.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.692+0000 7f038931e640 1 -- 172.21.15.60:0/1689778663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038406d300 msgr2=0x7f038406e9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:00.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.692+0000 7f038931e640 1 --2- 172.21.15.60:0/1689778663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038406d300 0x7f038406e9f0 crc :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:00.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.692+0000 7f038931e640 1 -- 172.21.15.60:0/1689778663 shutdown_connections 2023-08-24T10:08:00.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.692+0000 7f038931e640 1 --2- 172.21.15.60:0/1689778663 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0364037130 0x7f0364039640 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:00.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.692+0000 7f038931e640 1 --2- 172.21.15.60:0/1689778663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038406d300 0x7f038406e9f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:00.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.692+0000 7f038931e640 1 -- 172.21.15.60:0/1689778663 >> 172.21.15.60:0/1689778663 conn(0x7f038409e110 msgr2=0x7f038409bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:00.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.692+0000 7f038931e640 1 -- 172.21.15.60:0/1689778663 shutdown_connections 2023-08-24T10:08:00.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:00.692+0000 7f038931e640 1 -- 172.21.15.60:0/1689778663 wait complete. 2023-08-24T10:08:00.707 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2023-08-24T10:07:59.530122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:03.710 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:03.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.830+0000 7f08dd39e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:03.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.834+0000 7f08dd39e640 1 Processor -- start 2023-08-24T10:08:03.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.834+0000 7f08dd39e640 1 -- start start 2023-08-24T10:08:03.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.835+0000 7f08dd39e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08d812d040 0x7f08d81358b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:03.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.835+0000 7f08dd39e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d80709e0 con 0x7f08d806fa30 2023-08-24T10:08:03.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.835+0000 7f08dd39e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d812d8d0 con 0x7f08d812d040 2023-08-24T10:08:03.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.835+0000 7f08ce7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08d812d040 0x7f08d81358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:03.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.835+0000 7f08ce7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08d812d040 0x7f08d81358b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44282/0 (socket says 172.21.15.60:44282) 2023-08-24T10:08:03.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.835+0000 7f08ce7fc640 1 -- 172.21.15.60:0/4111606072 learned_addr learned my addr 172.21.15.60:0/4111606072 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:03.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.835+0000 7f08d6ffd640 1 --1- 172.21.15.60:0/4111606072 >> v1:172.21.15.60:6789/0 conn(0x7f08d806fa30 0x7f08d8098cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46932/0 (socket says 172.21.15.60:46932) 2023-08-24T10:08:03.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.835+0000 7f08ce7fc640 1 -- 172.21.15.60:0/4111606072 >> v1:172.21.15.60:6789/0 conn(0x7f08d806fa30 legacy=0x7f08d8098cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:03.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.835+0000 7f08ce7fc640 1 -- 172.21.15.60:0/4111606072 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d812d5c0 con 0x7f08d812d040 2023-08-24T10:08:03.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.835+0000 7f08ce7fc640 1 --2- 172.21.15.60:0/4111606072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08d812d040 0x7f08d81358b0 crc :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945abd6751291a23 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:03.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.835+0000 7f08d67fc640 1 -- 172.21.15.60:0/4111606072 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f08c0009dd0 con 0x7f08d812d040 2023-08-24T10:08:03.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.835+0000 7f08d67fc640 1 -- 172.21.15.60:0/4111606072 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f08c000e230 con 0x7f08d812d040 2023-08-24T10:08:03.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.835+0000 7f08d67fc640 1 -- 172.21.15.60:0/4111606072 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f08c000e4d0 con 0x7f08d812d040 2023-08-24T10:08:03.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.836+0000 7f08dd39e640 1 -- 172.21.15.60:0/4111606072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08d812d040 msgr2=0x7f08d81358b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:03.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.836+0000 7f08dd39e640 1 --2- 172.21.15.60:0/4111606072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08d812d040 0x7f08d81358b0 crc :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:03.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.836+0000 7f08dd39e640 1 -- 172.21.15.60:0/4111606072 shutdown_connections 2023-08-24T10:08:03.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.836+0000 7f08dd39e640 1 --2- 172.21.15.60:0/4111606072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08d812d040 0x7f08d81358b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:03.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.836+0000 7f08dd39e640 1 -- 172.21.15.60:0/4111606072 >> 172.21.15.60:0/4111606072 conn(0x7f08d80a2700 msgr2=0x7f08d80a2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:03.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.836+0000 7f08dd39e640 1 -- 172.21.15.60:0/4111606072 shutdown_connections 2023-08-24T10:08:03.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.836+0000 7f08dd39e640 1 -- 172.21.15.60:0/4111606072 wait complete. 2023-08-24T10:08:03.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.836+0000 7f08dd39e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:03.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.837+0000 7f08dd39e640 1 Processor -- start 2023-08-24T10:08:03.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.838+0000 7f08dd39e640 1 -- start start 2023-08-24T10:08:03.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.838+0000 7f08dd39e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08d806fa30 0x7f08d8119190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:03.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.838+0000 7f08dd39e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d81b9c00 con 0x7f08d806fa30 2023-08-24T10:08:03.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.838+0000 7f08d6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08d806fa30 0x7f08d8119190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:03.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.838+0000 7f08d6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08d806fa30 0x7f08d8119190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44284/0 (socket says 172.21.15.60:44284) 2023-08-24T10:08:03.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.838+0000 7f08d6ffd640 1 -- 172.21.15.60:0/676526025 learned_addr learned my addr 172.21.15.60:0/676526025 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:03.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.839+0000 7f08d6ffd640 1 -- 172.21.15.60:0/676526025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d8119cf0 con 0x7f08d806fa30 2023-08-24T10:08:03.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.839+0000 7f08d6ffd640 1 --2- 172.21.15.60:0/676526025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08d806fa30 0x7f08d8119190 crc :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:03.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.839+0000 7f08cffff640 1 -- 172.21.15.60:0/676526025 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f08c80071e0 con 0x7f08d806fa30 2023-08-24T10:08:03.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.839+0000 7f08cffff640 1 -- 172.21.15.60:0/676526025 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f08c8007340 con 0x7f08d806fa30 2023-08-24T10:08:03.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.839+0000 7f08dd39e640 1 -- 172.21.15.60:0/676526025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d8130980 con 0x7f08d806fa30 2023-08-24T10:08:03.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.839+0000 7f08cffff640 1 -- 172.21.15.60:0/676526025 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f08c80075e0 con 0x7f08d806fa30 2023-08-24T10:08:03.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.839+0000 7f08dd39e640 1 -- 172.21.15.60:0/676526025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d8119820 con 0x7f08d806fa30 2023-08-24T10:08:03.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.840+0000 7f08cffff640 1 -- 172.21.15.60:0/676526025 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f08c80057e0 con 0x7f08d806fa30 2023-08-24T10:08:03.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.840+0000 7f08dd39e640 1 -- 172.21.15.60:0/676526025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d8061800 con 0x7f08d806fa30 2023-08-24T10:08:03.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.840+0000 7f08cffff640 1 --2- 172.21.15.60:0/676526025 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f08b80370e0 0x7f08b80395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:03.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.840+0000 7f08cffff640 1 -- 172.21.15.60:0/676526025 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f08c80462d0 con 0x7f08d806fa30 2023-08-24T10:08:03.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.840+0000 7f08ce7fc640 1 --2- 172.21.15.60:0/676526025 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f08b80370e0 0x7f08b80395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:03.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.841+0000 7f08ce7fc640 1 --2- 172.21.15.60:0/676526025 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f08b80370e0 0x7f08b80395f0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:03.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:03.845+0000 7f08cffff640 1 -- 172.21.15.60:0/676526025 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f08c800d330 con 0x7f08d806fa30 2023-08-24T10:08:04.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.017+0000 7f08dd39e640 1 -- 172.21.15.60:0/676526025 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f08d81310a0 con 0x7f08b80370e0 2023-08-24T10:08:04.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.019+0000 7f08cffff640 1 -- 172.21.15.60:0/676526025 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f08d81310a0 con 0x7f08b80370e0 2023-08-24T10:08:04.021 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:04.023 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:04.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.020+0000 7f08dd39e640 1 -- 172.21.15.60:0/676526025 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f08b80370e0 msgr2=0x7f08b80395f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:04.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.020+0000 7f08dd39e640 1 --2- 172.21.15.60:0/676526025 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f08b80370e0 0x7f08b80395f0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:04.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.020+0000 7f08dd39e640 1 -- 172.21.15.60:0/676526025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08d806fa30 msgr2=0x7f08d8119190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:04.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.020+0000 7f08dd39e640 1 --2- 172.21.15.60:0/676526025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08d806fa30 0x7f08d8119190 crc :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:04.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.020+0000 7f08dd39e640 1 -- 172.21.15.60:0/676526025 shutdown_connections 2023-08-24T10:08:04.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.020+0000 7f08dd39e640 1 --2- 172.21.15.60:0/676526025 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f08b80370e0 0x7f08b80395f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:04.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.020+0000 7f08dd39e640 1 --2- 172.21.15.60:0/676526025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08d806fa30 0x7f08d8119190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:04.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.020+0000 7f08dd39e640 1 -- 172.21.15.60:0/676526025 >> 172.21.15.60:0/676526025 conn(0x7f08d80a2700 msgr2=0x7f08d81b87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:04.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.020+0000 7f08dd39e640 1 -- 172.21.15.60:0/676526025 shutdown_connections 2023-08-24T10:08:04.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.020+0000 7f08dd39e640 1 -- 172.21.15.60:0/676526025 wait complete. 2023-08-24T10:08:04.033 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2023-08-24T10:08:03.530688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:04.034 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:04.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.140+0000 7fe18506c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:04.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.144+0000 7fe18506c640 1 Processor -- start 2023-08-24T10:08:04.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.144+0000 7fe18506c640 1 -- start start 2023-08-24T10:08:04.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.145+0000 7fe18506c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe18006fa30 0x7fe180098cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:04.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.145+0000 7fe18506c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1800709e0 con 0x7fe18012d040 2023-08-24T10:08:04.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.145+0000 7fe18506c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe180099230 con 0x7fe18006fa30 2023-08-24T10:08:04.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.145+0000 7fe17ed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe18006fa30 0x7fe180098cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:04.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.145+0000 7fe17ed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe18006fa30 0x7fe180098cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44298/0 (socket says 172.21.15.60:44298) 2023-08-24T10:08:04.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.145+0000 7fe17ed9d640 1 -- 172.21.15.60:0/333561790 learned_addr learned my addr 172.21.15.60:0/333561790 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:04.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.145+0000 7fe17ed9d640 1 -- 172.21.15.60:0/333561790 >> v1:172.21.15.60:6789/0 conn(0x7fe18012d040 legacy=0x7fe180130530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:04.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.145+0000 7fe17ed9d640 1 -- 172.21.15.60:0/333561790 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18012d8d0 con 0x7fe18006fa30 2023-08-24T10:08:04.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.146+0000 7fe17ed9d640 1 --2- 172.21.15.60:0/333561790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe18006fa30 0x7fe180098cf0 crc :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc17b41600041e7c server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:04.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.146+0000 7fe17dd9b640 1 -- 172.21.15.60:0/333561790 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe178009dd0 con 0x7fe18006fa30 2023-08-24T10:08:04.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.146+0000 7fe17dd9b640 1 -- 172.21.15.60:0/333561790 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fe17800e230 con 0x7fe18006fa30 2023-08-24T10:08:04.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.146+0000 7fe17dd9b640 1 -- 172.21.15.60:0/333561790 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe17800e4d0 con 0x7fe18006fa30 2023-08-24T10:08:04.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.146+0000 7fe18506c640 1 -- 172.21.15.60:0/333561790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe18006fa30 msgr2=0x7fe180098cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:04.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.146+0000 7fe18506c640 1 --2- 172.21.15.60:0/333561790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe18006fa30 0x7fe180098cf0 crc :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:04.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.146+0000 7fe18506c640 1 -- 172.21.15.60:0/333561790 shutdown_connections 2023-08-24T10:08:04.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.146+0000 7fe18506c640 1 --2- 172.21.15.60:0/333561790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe18006fa30 0x7fe180098cf0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:04.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.146+0000 7fe18506c640 1 -- 172.21.15.60:0/333561790 >> 172.21.15.60:0/333561790 conn(0x7fe1800a2700 msgr2=0x7fe1800a2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:04.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.147+0000 7fe18506c640 1 -- 172.21.15.60:0/333561790 shutdown_connections 2023-08-24T10:08:04.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.147+0000 7fe18506c640 1 -- 172.21.15.60:0/333561790 wait complete. 2023-08-24T10:08:04.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.147+0000 7fe18506c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:04.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.148+0000 7fe18506c640 1 Processor -- start 2023-08-24T10:08:04.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.148+0000 7fe18506c640 1 -- start start 2023-08-24T10:08:04.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.149+0000 7fe18506c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe18006fa30 0x7fe18009aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:04.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.149+0000 7fe18506c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18009c5a0 con 0x7fe18006fa30 2023-08-24T10:08:04.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.149+0000 7fe17ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe18006fa30 0x7fe18009aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:04.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.149+0000 7fe17ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe18006fa30 0x7fe18009aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44304/0 (socket says 172.21.15.60:44304) 2023-08-24T10:08:04.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.149+0000 7fe17ed9d640 1 -- 172.21.15.60:0/1786257357 learned_addr learned my addr 172.21.15.60:0/1786257357 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:04.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.149+0000 7fe17ed9d640 1 -- 172.21.15.60:0/1786257357 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18009a580 con 0x7fe18006fa30 2023-08-24T10:08:04.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.149+0000 7fe17ed9d640 1 --2- 172.21.15.60:0/1786257357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe18006fa30 0x7fe18009aad0 crc :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:04.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.149+0000 7fe1677fe640 1 -- 172.21.15.60:0/1786257357 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe178014410 con 0x7fe18006fa30 2023-08-24T10:08:04.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.149+0000 7fe1677fe640 1 -- 172.21.15.60:0/1786257357 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fe178014570 con 0x7fe18006fa30 2023-08-24T10:08:04.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.149+0000 7fe18506c640 1 -- 172.21.15.60:0/1786257357 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18006d3c0 con 0x7fe18006fa30 2023-08-24T10:08:04.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.149+0000 7fe1677fe640 1 -- 172.21.15.60:0/1786257357 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe178014810 con 0x7fe18006fa30 2023-08-24T10:08:04.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.149+0000 7fe18506c640 1 -- 172.21.15.60:0/1786257357 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe180118a10 con 0x7fe18006fa30 2023-08-24T10:08:04.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.150+0000 7fe1677fe640 1 -- 172.21.15.60:0/1786257357 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fe1780149c0 con 0x7fe18006fa30 2023-08-24T10:08:04.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.150+0000 7fe1677fe640 1 --2- 172.21.15.60:0/1786257357 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe16c037130 0x7fe16c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:04.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.150+0000 7fe18506c640 1 -- 172.21.15.60:0/1786257357 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe180061800 con 0x7fe18006fa30 2023-08-24T10:08:04.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.150+0000 7fe1677fe640 1 -- 172.21.15.60:0/1786257357 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fe178019040 con 0x7fe18006fa30 2023-08-24T10:08:04.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.151+0000 7fe17e59c640 1 --2- 172.21.15.60:0/1786257357 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe16c037130 0x7fe16c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:04.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.151+0000 7fe17e59c640 1 --2- 172.21.15.60:0/1786257357 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe16c037130 0x7fe16c039640 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:04.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.155+0000 7fe1677fe640 1 -- 172.21.15.60:0/1786257357 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fe17800f8f0 con 0x7fe18006fa30 2023-08-24T10:08:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.327+0000 7fe18506c640 1 -- 172.21.15.60:0/1786257357 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe18009a410 con 0x7fe16c037130 2023-08-24T10:08:04.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.329+0000 7fe1677fe640 1 -- 172.21.15.60:0/1786257357 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7fe18009a410 con 0x7fe16c037130 2023-08-24T10:08:04.331 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:04.332 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:04.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.330+0000 7fe18506c640 1 -- 172.21.15.60:0/1786257357 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe16c037130 msgr2=0x7fe16c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:04.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.330+0000 7fe18506c640 1 --2- 172.21.15.60:0/1786257357 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe16c037130 0x7fe16c039640 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:04.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.330+0000 7fe18506c640 1 -- 172.21.15.60:0/1786257357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe18006fa30 msgr2=0x7fe18009aad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:04.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.330+0000 7fe18506c640 1 --2- 172.21.15.60:0/1786257357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe18006fa30 0x7fe18009aad0 crc :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:04.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.330+0000 7fe18506c640 1 -- 172.21.15.60:0/1786257357 shutdown_connections 2023-08-24T10:08:04.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.330+0000 7fe18506c640 1 --2- 172.21.15.60:0/1786257357 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe16c037130 0x7fe16c039640 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:04.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.330+0000 7fe18506c640 1 --2- 172.21.15.60:0/1786257357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe18006fa30 0x7fe18009aad0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:04.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.330+0000 7fe18506c640 1 -- 172.21.15.60:0/1786257357 >> 172.21.15.60:0/1786257357 conn(0x7fe1800a2700 msgr2=0x7fe1801b8630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:04.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.330+0000 7fe18506c640 1 -- 172.21.15.60:0/1786257357 shutdown_connections 2023-08-24T10:08:04.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.330+0000 7fe18506c640 1 -- 172.21.15.60:0/1786257357 wait complete. 2023-08-24T10:08:04.343 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2023-08-24T10:08:03.530688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:04.344 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:08:04.344 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:04.344 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:08:04.345 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:04.345 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:08:04.345 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:04.345 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:08:04.345 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:04.346 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:08:04.346 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:04.346 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:08:04.346 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:04.347 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:08:04.347 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:04.347 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:08:04.347 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:04.348 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:08:04.348 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:04.348 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:08:04.380 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~177s 2023-08-24T10:08:04.380 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~177s 2023-08-24T10:08:04.381 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~177s 2023-08-24T10:08:04.381 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~177s 2023-08-24T10:08:04.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.454+0000 7f8c8c430640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:04.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.457+0000 7f8c8c430640 1 Processor -- start 2023-08-24T10:08:04.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.457+0000 7f8c8c430640 1 -- start start 2023-08-24T10:08:04.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.458+0000 7f8c8c430640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8c8412f610 0x7f8c8412fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:04.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.458+0000 7f8c8c430640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8409d750 con 0x7f8c841326f0 2023-08-24T10:08:04.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.458+0000 7f8c8c430640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c8409d980 con 0x7f8c8412f610 2023-08-24T10:08:04.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.458+0000 7f8c8a1cc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8c8412f610 0x7f8c8412fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:04.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.458+0000 7f8c8a1cc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8c8412f610 0x7f8c8412fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44312/0 (socket says 172.21.15.60:44312) 2023-08-24T10:08:04.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.458+0000 7f8c8a1cc640 1 -- 172.21.15.60:0/3944172956 learned_addr learned my addr 172.21.15.60:0/3944172956 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:04.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.458+0000 7f8c8a1cc640 1 -- 172.21.15.60:0/3944172956 >> v1:172.21.15.60:6789/0 conn(0x7f8c841326f0 legacy=0x7f8c84132b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:04.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.458+0000 7f8c8a1cc640 1 -- 172.21.15.60:0/3944172956 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8409bde0 con 0x7f8c8412f610 2023-08-24T10:08:04.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.458+0000 7f8c8a1cc640 1 --2- 172.21.15.60:0/3944172956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c8412f610 0x7f8c8412fa20 crc :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc6780280c8a973 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:04.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.458+0000 7f8c891ca640 1 -- 172.21.15.60:0/3944172956 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8c80009dc0 con 0x7f8c8412f610 2023-08-24T10:08:04.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.458+0000 7f8c891ca640 1 -- 172.21.15.60:0/3944172956 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f8c8000e230 con 0x7f8c8412f610 2023-08-24T10:08:04.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.458+0000 7f8c891ca640 1 -- 172.21.15.60:0/3944172956 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8c8000e4d0 con 0x7f8c8412f610 2023-08-24T10:08:04.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.459+0000 7f8c8c430640 1 -- 172.21.15.60:0/3944172956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c8412f610 msgr2=0x7f8c8412fa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:04.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.459+0000 7f8c8c430640 1 --2- 172.21.15.60:0/3944172956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c8412f610 0x7f8c8412fa20 crc :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:04.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.459+0000 7f8c8c430640 1 -- 172.21.15.60:0/3944172956 shutdown_connections 2023-08-24T10:08:04.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.459+0000 7f8c8c430640 1 --2- 172.21.15.60:0/3944172956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c8412f610 0x7f8c8412fa20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:04.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.459+0000 7f8c8c430640 1 -- 172.21.15.60:0/3944172956 >> 172.21.15.60:0/3944172956 conn(0x7f8c840a2500 msgr2=0x7f8c840a2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:04.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.459+0000 7f8c8c430640 1 -- 172.21.15.60:0/3944172956 shutdown_connections 2023-08-24T10:08:04.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.459+0000 7f8c8c430640 1 -- 172.21.15.60:0/3944172956 wait complete. 2023-08-24T10:08:04.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.459+0000 7f8c8c430640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:04.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.460+0000 7f8c8c430640 1 Processor -- start 2023-08-24T10:08:04.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.460+0000 7f8c8c430640 1 -- start start 2023-08-24T10:08:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.461+0000 7f8c8c430640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c8412f610 0x7f8c8409fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.461+0000 7f8c8c430640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8412cc60 con 0x7f8c8412f610 2023-08-24T10:08:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.461+0000 7f8c8a1cc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c8412f610 0x7f8c8409fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:04.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.461+0000 7f8c8a1cc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c8412f610 0x7f8c8409fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44318/0 (socket says 172.21.15.60:44318) 2023-08-24T10:08:04.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.461+0000 7f8c8a1cc640 1 -- 172.21.15.60:0/932093544 learned_addr learned my addr 172.21.15.60:0/932093544 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:04.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.461+0000 7f8c8a1cc640 1 -- 172.21.15.60:0/932093544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8406a1f0 con 0x7f8c8412f610 2023-08-24T10:08:04.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.461+0000 7f8c8a1cc640 1 --2- 172.21.15.60:0/932093544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c8412f610 0x7f8c8409fca0 crc :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:04.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.461+0000 7f8c7affd640 1 -- 172.21.15.60:0/932093544 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8c80014540 con 0x7f8c8412f610 2023-08-24T10:08:04.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.461+0000 7f8c7affd640 1 -- 172.21.15.60:0/932093544 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f8c800146a0 con 0x7f8c8412f610 2023-08-24T10:08:04.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.462+0000 7f8c8c430640 1 -- 172.21.15.60:0/932093544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8409f7e0 con 0x7f8c8412f610 2023-08-24T10:08:04.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.462+0000 7f8c7affd640 1 -- 172.21.15.60:0/932093544 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8c80014940 con 0x7f8c8412f610 2023-08-24T10:08:04.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.462+0000 7f8c8c430640 1 -- 172.21.15.60:0/932093544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8409f4d0 con 0x7f8c8412f610 2023-08-24T10:08:04.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.462+0000 7f8c7affd640 1 -- 172.21.15.60:0/932093544 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f8c80014af0 con 0x7f8c8412f610 2023-08-24T10:08:04.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.462+0000 7f8c8c430640 1 -- 172.21.15.60:0/932093544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c84061800 con 0x7f8c8412f610 2023-08-24T10:08:04.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.463+0000 7f8c7affd640 1 --2- 172.21.15.60:0/932093544 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8c68037130 0x7f8c68039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:04.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.463+0000 7f8c7affd640 1 -- 172.21.15.60:0/932093544 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f8c80019040 con 0x7f8c8412f610 2023-08-24T10:08:04.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.463+0000 7f8c899cb640 1 --2- 172.21.15.60:0/932093544 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8c68037130 0x7f8c68039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:04.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.463+0000 7f8c899cb640 1 --2- 172.21.15.60:0/932093544 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8c68037130 0x7f8c68039640 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:04.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.467+0000 7f8c7affd640 1 -- 172.21.15.60:0/932093544 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f8c8000fb00 con 0x7f8c8412f610 2023-08-24T10:08:04.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.711+0000 7f8c8c430640 1 -- 172.21.15.60:0/932093544 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8c8406a1f0 con 0x7f8c8412f610 2023-08-24T10:08:04.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.711+0000 7f8c7affd640 1 -- 172.21.15.60:0/932093544 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f8c8406a1f0 con 0x7f8c8412f610 2023-08-24T10:08:04.714 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:04.714 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":195,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:08:04.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.713+0000 7f8c8c430640 1 -- 172.21.15.60:0/932093544 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8c68037130 msgr2=0x7f8c68039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:04.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.713+0000 7f8c8c430640 1 --2- 172.21.15.60:0/932093544 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8c68037130 0x7f8c68039640 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:04.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.713+0000 7f8c8c430640 1 -- 172.21.15.60:0/932093544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c8412f610 msgr2=0x7f8c8409fca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:04.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.713+0000 7f8c8c430640 1 --2- 172.21.15.60:0/932093544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c8412f610 0x7f8c8409fca0 crc :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:04.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.713+0000 7f8c8c430640 1 -- 172.21.15.60:0/932093544 shutdown_connections 2023-08-24T10:08:04.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.713+0000 7f8c8c430640 1 --2- 172.21.15.60:0/932093544 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8c68037130 0x7f8c68039640 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:04.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.713+0000 7f8c8c430640 1 --2- 172.21.15.60:0/932093544 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c8412f610 0x7f8c8409fca0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:04.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.713+0000 7f8c8c430640 1 -- 172.21.15.60:0/932093544 >> 172.21.15.60:0/932093544 conn(0x7f8c840a2500 msgr2=0x7f8c841ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:04.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.713+0000 7f8c8c430640 1 -- 172.21.15.60:0/932093544 shutdown_connections 2023-08-24T10:08:04.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.713+0000 7f8c8c430640 1 -- 172.21.15.60:0/932093544 wait complete. 2023-08-24T10:08:04.726 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:08:04.727 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:04.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.835+0000 7fd92f59e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:04.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.839+0000 7fd92f59e640 1 Processor -- start 2023-08-24T10:08:04.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.839+0000 7fd92f59e640 1 -- start start 2023-08-24T10:08:04.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.840+0000 7fd92f59e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd928070110 0x7fd9281356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:04.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.840+0000 7fd92f59e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd928096f10 con 0x7fd928071e40 2023-08-24T10:08:04.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.840+0000 7fd92f59e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9280977f0 con 0x7fd928070110 2023-08-24T10:08:04.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.840+0000 7fd92cb39640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd928070110 0x7fd9281356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:04.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.840+0000 7fd92cb39640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd928070110 0x7fd9281356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44328/0 (socket says 172.21.15.60:44328) 2023-08-24T10:08:04.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.840+0000 7fd92cb39640 1 -- 172.21.15.60:0/282050293 learned_addr learned my addr 172.21.15.60:0/282050293 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:04.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.840+0000 7fd92d33a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd928071e40 0x7fd928096800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46954/0 (socket says 172.21.15.60:46954) 2023-08-24T10:08:04.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.840+0000 7fd92cb39640 1 -- 172.21.15.60:0/282050293 >> v1:172.21.15.60:6789/0 conn(0x7fd928071e40 legacy=0x7fd928096800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:04.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.840+0000 7fd92cb39640 1 -- 172.21.15.60:0/282050293 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92809a280 con 0x7fd928070110 2023-08-24T10:08:04.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.840+0000 7fd92cb39640 1 --2- 172.21.15.60:0/282050293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd928070110 0x7fd9281356a0 crc :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1fceaa85f60fab server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:04.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.840+0000 7fd91ffff640 1 -- 172.21.15.60:0/282050293 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd918009dd0 con 0x7fd928070110 2023-08-24T10:08:04.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.840+0000 7fd91ffff640 1 -- 172.21.15.60:0/282050293 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd91800e230 con 0x7fd928070110 2023-08-24T10:08:04.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.840+0000 7fd91ffff640 1 -- 172.21.15.60:0/282050293 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd91800e4d0 con 0x7fd928070110 2023-08-24T10:08:04.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.841+0000 7fd92f59e640 1 -- 172.21.15.60:0/282050293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd928070110 msgr2=0x7fd9281356a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:04.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.841+0000 7fd92f59e640 1 --2- 172.21.15.60:0/282050293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd928070110 0x7fd9281356a0 crc :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:04.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.841+0000 7fd92f59e640 1 -- 172.21.15.60:0/282050293 shutdown_connections 2023-08-24T10:08:04.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.841+0000 7fd92f59e640 1 --2- 172.21.15.60:0/282050293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd928070110 0x7fd9281356a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:04.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.841+0000 7fd92f59e640 1 -- 172.21.15.60:0/282050293 >> 172.21.15.60:0/282050293 conn(0x7fd92809e110 msgr2=0x7fd92809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:04.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.841+0000 7fd92f59e640 1 -- 172.21.15.60:0/282050293 shutdown_connections 2023-08-24T10:08:04.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.841+0000 7fd92f59e640 1 -- 172.21.15.60:0/282050293 wait complete. 2023-08-24T10:08:04.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.841+0000 7fd92f59e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:04.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.842+0000 7fd92f59e640 1 Processor -- start 2023-08-24T10:08:04.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.842+0000 7fd92f59e640 1 -- start start 2023-08-24T10:08:04.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.843+0000 7fd92f59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd928070110 0x7fd928093460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:04.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.843+0000 7fd92f59e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92808ff10 con 0x7fd928070110 2023-08-24T10:08:04.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.843+0000 7fd92d33a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd928070110 0x7fd928093460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:04.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.843+0000 7fd92d33a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd928070110 0x7fd928093460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44336/0 (socket says 172.21.15.60:44336) 2023-08-24T10:08:04.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.843+0000 7fd92d33a640 1 -- 172.21.15.60:0/1587939425 learned_addr learned my addr 172.21.15.60:0/1587939425 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:04.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.843+0000 7fd92d33a640 1 -- 172.21.15.60:0/1587939425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd928090680 con 0x7fd928070110 2023-08-24T10:08:04.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.843+0000 7fd92d33a640 1 --2- 172.21.15.60:0/1587939425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd928070110 0x7fd928093460 crc :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:04.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.843+0000 7fd91dffb640 1 -- 172.21.15.60:0/1587939425 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd9240071e0 con 0x7fd928070110 2023-08-24T10:08:04.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.844+0000 7fd91dffb640 1 -- 172.21.15.60:0/1587939425 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd924007340 con 0x7fd928070110 2023-08-24T10:08:04.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.844+0000 7fd92f59e640 1 -- 172.21.15.60:0/1587939425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd928092e80 con 0x7fd928070110 2023-08-24T10:08:04.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.844+0000 7fd91dffb640 1 -- 172.21.15.60:0/1587939425 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd9240075e0 con 0x7fd928070110 2023-08-24T10:08:04.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.844+0000 7fd92f59e640 1 -- 172.21.15.60:0/1587939425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd928092b70 con 0x7fd928070110 2023-08-24T10:08:04.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.844+0000 7fd91dffb640 1 -- 172.21.15.60:0/1587939425 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fd9240057e0 con 0x7fd928070110 2023-08-24T10:08:04.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.845+0000 7fd91dffb640 1 --2- 172.21.15.60:0/1587939425 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd90c037130 0x7fd90c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:04.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.845+0000 7fd91dffb640 1 -- 172.21.15.60:0/1587939425 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fd9240463b0 con 0x7fd928070110 2023-08-24T10:08:04.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.845+0000 7fd92f59e640 1 -- 172.21.15.60:0/1587939425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd928061800 con 0x7fd928070110 2023-08-24T10:08:04.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.845+0000 7fd92cb39640 1 --2- 172.21.15.60:0/1587939425 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd90c037130 0x7fd90c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:04.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.845+0000 7fd92cb39640 1 --2- 172.21.15.60:0/1587939425 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd90c037130 0x7fd90c039640 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:04.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:04.850+0000 7fd91dffb640 1 -- 172.21.15.60:0/1587939425 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fd92400d330 con 0x7fd928070110 2023-08-24T10:08:05.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:05.023+0000 7fd92f59e640 1 -- 172.21.15.60:0/1587939425 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd9280932f0 con 0x7fd90c037130 2023-08-24T10:08:05.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:05.025+0000 7fd91dffb640 1 -- 172.21.15.60:0/1587939425 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7fd9280932f0 con 0x7fd90c037130 2023-08-24T10:08:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:05.027 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:05.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:05.026+0000 7fd92f59e640 1 -- 172.21.15.60:0/1587939425 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd90c037130 msgr2=0x7fd90c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:05.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:05.026+0000 7fd92f59e640 1 --2- 172.21.15.60:0/1587939425 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd90c037130 0x7fd90c039640 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:05.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:05.026+0000 7fd92f59e640 1 -- 172.21.15.60:0/1587939425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd928070110 msgr2=0x7fd928093460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:05.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:05.026+0000 7fd92f59e640 1 --2- 172.21.15.60:0/1587939425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd928070110 0x7fd928093460 crc :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:05.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:05.026+0000 7fd92f59e640 1 -- 172.21.15.60:0/1587939425 shutdown_connections 2023-08-24T10:08:05.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:05.026+0000 7fd92f59e640 1 --2- 172.21.15.60:0/1587939425 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd90c037130 0x7fd90c039640 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:05.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:05.026+0000 7fd92f59e640 1 --2- 172.21.15.60:0/1587939425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd928070110 0x7fd928093460 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:05.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:05.026+0000 7fd92f59e640 1 -- 172.21.15.60:0/1587939425 >> 172.21.15.60:0/1587939425 conn(0x7fd92809e110 msgr2=0x7fd928098490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:05.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:05.026+0000 7fd92f59e640 1 -- 172.21.15.60:0/1587939425 shutdown_connections 2023-08-24T10:08:05.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:05.026+0000 7fd92f59e640 1 -- 172.21.15.60:0/1587939425 wait complete. 2023-08-24T10:08:05.039 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2023-08-24T10:08:03.530688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:08.040 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:08.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.160+0000 7f56f08ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:08.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.164+0000 7f56f08ac640 1 Processor -- start 2023-08-24T10:08:08.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.165+0000 7f56f08ac640 1 -- start start 2023-08-24T10:08:08.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.165+0000 7f56f08ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f56ec096860 0x7f56ec096c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:08.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.165+0000 7f56f08ac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ec0971b0 con 0x7f56ec0990b0 2023-08-24T10:08:08.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.165+0000 7f56f08ac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ec06db70 con 0x7f56ec096860 2023-08-24T10:08:08.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.165+0000 7f56ea59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f56ec096860 0x7f56ec096c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:08.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.165+0000 7f56ea59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f56ec096860 0x7f56ec096c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32870/0 (socket says 172.21.15.60:32870) 2023-08-24T10:08:08.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.165+0000 7f56ea59c640 1 -- 172.21.15.60:0/2378154692 learned_addr learned my addr 172.21.15.60:0/2378154692 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:08.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.166+0000 7f56ea59c640 1 -- 172.21.15.60:0/2378154692 >> v1:172.21.15.60:6789/0 conn(0x7f56ec0990b0 legacy=0x7f56ec0994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:08.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.166+0000 7f56ea59c640 1 -- 172.21.15.60:0/2378154692 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec06e320 con 0x7f56ec096860 2023-08-24T10:08:08.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.166+0000 7f56ea59c640 1 --2- 172.21.15.60:0/2378154692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56ec096860 0x7f56ec096c70 crc :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff196661bcbf9622 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:08.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.166+0000 7f56e9d9b640 1 -- 172.21.15.60:0/2378154692 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f56dc009dd0 con 0x7f56ec096860 2023-08-24T10:08:08.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.166+0000 7f56e9d9b640 1 -- 172.21.15.60:0/2378154692 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f56dc00e230 con 0x7f56ec096860 2023-08-24T10:08:08.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.166+0000 7f56e9d9b640 1 -- 172.21.15.60:0/2378154692 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f56dc00e4d0 con 0x7f56ec096860 2023-08-24T10:08:08.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.167+0000 7f56f08ac640 1 -- 172.21.15.60:0/2378154692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56ec096860 msgr2=0x7f56ec096c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:08.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.167+0000 7f56f08ac640 1 --2- 172.21.15.60:0/2378154692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56ec096860 0x7f56ec096c70 crc :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:08.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.167+0000 7f56f08ac640 1 -- 172.21.15.60:0/2378154692 shutdown_connections 2023-08-24T10:08:08.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.167+0000 7f56f08ac640 1 --2- 172.21.15.60:0/2378154692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56ec096860 0x7f56ec096c70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:08.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.167+0000 7f56f08ac640 1 -- 172.21.15.60:0/2378154692 >> 172.21.15.60:0/2378154692 conn(0x7f56ec09e110 msgr2=0x7f56ec09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:08.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.167+0000 7f56f08ac640 1 -- 172.21.15.60:0/2378154692 shutdown_connections 2023-08-24T10:08:08.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.167+0000 7f56f08ac640 1 -- 172.21.15.60:0/2378154692 wait complete. 2023-08-24T10:08:08.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.167+0000 7f56f08ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:08.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.168+0000 7f56f08ac640 1 Processor -- start 2023-08-24T10:08:08.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.168+0000 7f56f08ac640 1 -- start start 2023-08-24T10:08:08.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.169+0000 7f56f08ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56ec096860 0x7f56ec06e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:08.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.169+0000 7f56f08ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec06fbb0 con 0x7f56ec096860 2023-08-24T10:08:08.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.169+0000 7f56ea59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56ec096860 0x7f56ec06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:08.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.169+0000 7f56ea59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56ec096860 0x7f56ec06e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32882/0 (socket says 172.21.15.60:32882) 2023-08-24T10:08:08.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.169+0000 7f56ea59c640 1 -- 172.21.15.60:0/2123974615 learned_addr learned my addr 172.21.15.60:0/2123974615 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:08.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.169+0000 7f56ea59c640 1 -- 172.21.15.60:0/2123974615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec071220 con 0x7f56ec096860 2023-08-24T10:08:08.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.169+0000 7f56ea59c640 1 --2- 172.21.15.60:0/2123974615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56ec096860 0x7f56ec06e720 crc :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:08.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.169+0000 7f56e2ffd640 1 -- 172.21.15.60:0/2123974615 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f56dc014410 con 0x7f56ec096860 2023-08-24T10:08:08.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.169+0000 7f56e2ffd640 1 -- 172.21.15.60:0/2123974615 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f56dc014570 con 0x7f56ec096860 2023-08-24T10:08:08.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.169+0000 7f56f08ac640 1 -- 172.21.15.60:0/2123974615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ec070f80 con 0x7f56ec096860 2023-08-24T10:08:08.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.169+0000 7f56e2ffd640 1 -- 172.21.15.60:0/2123974615 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f56dc014810 con 0x7f56ec096860 2023-08-24T10:08:08.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.169+0000 7f56f08ac640 1 -- 172.21.15.60:0/2123974615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ec06b0e0 con 0x7f56ec096860 2023-08-24T10:08:08.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.170+0000 7f56e2ffd640 1 -- 172.21.15.60:0/2123974615 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f56dc0149c0 con 0x7f56ec096860 2023-08-24T10:08:08.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.170+0000 7f56e2ffd640 1 --2- 172.21.15.60:0/2123974615 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f56cc037130 0x7f56cc039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:08.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.170+0000 7f56f08ac640 1 -- 172.21.15.60:0/2123974615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56ec070020 con 0x7f56ec096860 2023-08-24T10:08:08.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.171+0000 7f56e2ffd640 1 -- 172.21.15.60:0/2123974615 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f56dc019040 con 0x7f56ec096860 2023-08-24T10:08:08.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.171+0000 7f56e3fff640 1 --2- 172.21.15.60:0/2123974615 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f56cc037130 0x7f56cc039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:08.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.171+0000 7f56e3fff640 1 --2- 172.21.15.60:0/2123974615 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f56cc037130 0x7f56cc039640 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:08.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.175+0000 7f56e2ffd640 1 -- 172.21.15.60:0/2123974615 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f56dc00fb20 con 0x7f56ec096860 2023-08-24T10:08:08.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.349+0000 7f56f08ac640 1 -- 172.21.15.60:0/2123974615 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f56ec06e5a0 con 0x7f56cc037130 2023-08-24T10:08:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.351+0000 7f56e2ffd640 1 -- 172.21.15.60:0/2123974615 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f56d4001650 con 0x7f56cc037130 2023-08-24T10:08:08.353 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:08.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.353+0000 7f56f08ac640 1 -- 172.21.15.60:0/2123974615 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f56cc037130 msgr2=0x7f56cc039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:08.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.353+0000 7f56f08ac640 1 --2- 172.21.15.60:0/2123974615 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f56cc037130 0x7f56cc039640 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:08.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.353+0000 7f56f08ac640 1 -- 172.21.15.60:0/2123974615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56ec096860 msgr2=0x7f56ec06e720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:08.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.353+0000 7f56f08ac640 1 --2- 172.21.15.60:0/2123974615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56ec096860 0x7f56ec06e720 crc :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:08.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.353+0000 7f56f08ac640 1 -- 172.21.15.60:0/2123974615 shutdown_connections 2023-08-24T10:08:08.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.353+0000 7f56f08ac640 1 --2- 172.21.15.60:0/2123974615 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f56cc037130 0x7f56cc039640 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:08.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.353+0000 7f56f08ac640 1 --2- 172.21.15.60:0/2123974615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56ec096860 0x7f56ec06e720 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:08.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.353+0000 7f56f08ac640 1 -- 172.21.15.60:0/2123974615 >> 172.21.15.60:0/2123974615 conn(0x7f56ec09e110 msgr2=0x7f56ec09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:08.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.353+0000 7f56f08ac640 1 -- 172.21.15.60:0/2123974615 shutdown_connections 2023-08-24T10:08:08.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.353+0000 7f56f08ac640 1 -- 172.21.15.60:0/2123974615 wait complete. 2023-08-24T10:08:08.366 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2023-08-24T10:08:07.531322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:08.367 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:08.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.473+0000 7f61af39e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:08.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.477+0000 7f61af39e640 1 Processor -- start 2023-08-24T10:08:08.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.477+0000 7f61af39e640 1 -- start start 2023-08-24T10:08:08.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.478+0000 7f61af39e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f61a8135980 0x7f61a8137da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:08.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.478+0000 7f61af39e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a8132610 con 0x7f61a8132b50 2023-08-24T10:08:08.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.478+0000 7f61af39e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a809c260 con 0x7f61a8135980 2023-08-24T10:08:08.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.478+0000 7f61ac939640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f61a8135980 0x7f61a8137da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:08.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.478+0000 7f61ac939640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f61a8135980 0x7f61a8137da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32896/0 (socket says 172.21.15.60:32896) 2023-08-24T10:08:08.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.478+0000 7f61ac939640 1 -- 172.21.15.60:0/1218811720 learned_addr learned my addr 172.21.15.60:0/1218811720 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:08.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.479+0000 7f61ac939640 1 -- 172.21.15.60:0/1218811720 >> v1:172.21.15.60:6789/0 conn(0x7f61a8132b50 legacy=0x7f61a8132f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:08.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.479+0000 7f61ac939640 1 -- 172.21.15.60:0/1218811720 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a8132840 con 0x7f61a8135980 2023-08-24T10:08:08.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.479+0000 7f61ac939640 1 --2- 172.21.15.60:0/1218811720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61a8135980 0x7f61a8137da0 crc :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d3662146611070 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:08.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.479+0000 7f619ffff640 1 -- 172.21.15.60:0/1218811720 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6198009dd0 con 0x7f61a8135980 2023-08-24T10:08:08.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.479+0000 7f619ffff640 1 -- 172.21.15.60:0/1218811720 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f619800e230 con 0x7f61a8135980 2023-08-24T10:08:08.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.479+0000 7f619ffff640 1 -- 172.21.15.60:0/1218811720 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f619800e4d0 con 0x7f61a8135980 2023-08-24T10:08:08.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.480+0000 7f61af39e640 1 -- 172.21.15.60:0/1218811720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61a8135980 msgr2=0x7f61a8137da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:08.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.480+0000 7f61af39e640 1 --2- 172.21.15.60:0/1218811720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61a8135980 0x7f61a8137da0 crc :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:08.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.480+0000 7f61af39e640 1 -- 172.21.15.60:0/1218811720 shutdown_connections 2023-08-24T10:08:08.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.480+0000 7f61af39e640 1 --2- 172.21.15.60:0/1218811720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61a8135980 0x7f61a8137da0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:08.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.480+0000 7f61af39e640 1 -- 172.21.15.60:0/1218811720 >> 172.21.15.60:0/1218811720 conn(0x7f61a80a26a0 msgr2=0x7f61a80a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:08.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.480+0000 7f61af39e640 1 -- 172.21.15.60:0/1218811720 shutdown_connections 2023-08-24T10:08:08.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.480+0000 7f61af39e640 1 -- 172.21.15.60:0/1218811720 wait complete. 2023-08-24T10:08:08.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.480+0000 7f61af39e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:08.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.481+0000 7f61af39e640 1 Processor -- start 2023-08-24T10:08:08.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.481+0000 7f61af39e640 1 -- start start 2023-08-24T10:08:08.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.482+0000 7f61af39e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61a8132b50 0x7f61a80a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:08.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.482+0000 7f61af39e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a80a1b60 con 0x7f61a8132b50 2023-08-24T10:08:08.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.482+0000 7f61ad13a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61a8132b50 0x7f61a80a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:08.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.482+0000 7f61ad13a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61a8132b50 0x7f61a80a0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32910/0 (socket says 172.21.15.60:32910) 2023-08-24T10:08:08.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.482+0000 7f61ad13a640 1 -- 172.21.15.60:0/2258299802 learned_addr learned my addr 172.21.15.60:0/2258299802 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:08.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.482+0000 7f61ad13a640 1 -- 172.21.15.60:0/2258299802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a809fab0 con 0x7f61a8132b50 2023-08-24T10:08:08.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.482+0000 7f61ad13a640 1 --2- 172.21.15.60:0/2258299802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61a8132b50 0x7f61a80a0000 crc :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:08.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.482+0000 7f619dffb640 1 -- 172.21.15.60:0/2258299802 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f61a400c860 con 0x7f61a8132b50 2023-08-24T10:08:08.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.482+0000 7f619dffb640 1 -- 172.21.15.60:0/2258299802 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f61a400c9c0 con 0x7f61a8132b50 2023-08-24T10:08:08.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.482+0000 7f61af39e640 1 -- 172.21.15.60:0/2258299802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a806a240 con 0x7f61a8132b50 2023-08-24T10:08:08.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.482+0000 7f619dffb640 1 -- 172.21.15.60:0/2258299802 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f61a400cc60 con 0x7f61a8132b50 2023-08-24T10:08:08.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.482+0000 7f61af39e640 1 -- 172.21.15.60:0/2258299802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a811ac50 con 0x7f61a8132b50 2023-08-24T10:08:08.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.483+0000 7f619dffb640 1 -- 172.21.15.60:0/2258299802 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f61a4011310 con 0x7f61a8132b50 2023-08-24T10:08:08.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.483+0000 7f619dffb640 1 --2- 172.21.15.60:0/2258299802 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f618c037130 0x7f618c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:08.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.483+0000 7f619dffb640 1 -- 172.21.15.60:0/2258299802 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f61a40468d0 con 0x7f61a8132b50 2023-08-24T10:08:08.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.483+0000 7f61af39e640 1 -- 172.21.15.60:0/2258299802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a806a240 con 0x7f61a8132b50 2023-08-24T10:08:08.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.484+0000 7f61ac939640 1 --2- 172.21.15.60:0/2258299802 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f618c037130 0x7f618c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:08.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.484+0000 7f61ac939640 1 --2- 172.21.15.60:0/2258299802 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f618c037130 0x7f618c039640 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:08.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.488+0000 7f619dffb640 1 -- 172.21.15.60:0/2258299802 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f61a806a240 con 0x7f61a8132b50 2023-08-24T10:08:08.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.660+0000 7f61af39e640 1 -- 172.21.15.60:0/2258299802 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f61a809f940 con 0x7f618c037130 2023-08-24T10:08:08.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.661+0000 7f619dffb640 1 -- 172.21.15.60:0/2258299802 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f61a809f940 con 0x7f618c037130 2023-08-24T10:08:08.664 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:08.664 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:08.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.663+0000 7f61af39e640 1 -- 172.21.15.60:0/2258299802 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f618c037130 msgr2=0x7f618c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:08.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.663+0000 7f61af39e640 1 --2- 172.21.15.60:0/2258299802 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f618c037130 0x7f618c039640 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:08.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.663+0000 7f61af39e640 1 -- 172.21.15.60:0/2258299802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61a8132b50 msgr2=0x7f61a80a0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:08.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.663+0000 7f61af39e640 1 --2- 172.21.15.60:0/2258299802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61a8132b50 0x7f61a80a0000 crc :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:08.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.663+0000 7f61af39e640 1 -- 172.21.15.60:0/2258299802 shutdown_connections 2023-08-24T10:08:08.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.663+0000 7f61af39e640 1 --2- 172.21.15.60:0/2258299802 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f618c037130 0x7f618c039640 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:08.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.663+0000 7f61af39e640 1 --2- 172.21.15.60:0/2258299802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61a8132b50 0x7f61a80a0000 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:08.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.663+0000 7f61af39e640 1 -- 172.21.15.60:0/2258299802 >> 172.21.15.60:0/2258299802 conn(0x7f61a80a26a0 msgr2=0x7f61a81baaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:08.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.663+0000 7f61af39e640 1 -- 172.21.15.60:0/2258299802 shutdown_connections 2023-08-24T10:08:08.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.663+0000 7f61af39e640 1 -- 172.21.15.60:0/2258299802 wait complete. 2023-08-24T10:08:08.676 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2023-08-24T10:08:07.531322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:08.677 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:08:08.677 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:08.677 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:08:08.678 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:08.678 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:08:08.678 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:08.678 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:08:08.679 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:08.679 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:08:08.680 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:08.680 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:08:08.681 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:08.681 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:08:08.681 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:08.682 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:08:08.682 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:08.683 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:08:08.683 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:08.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:08:08.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.790+0000 7f06aed55640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:08.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.793+0000 7f06aed55640 1 Processor -- start 2023-08-24T10:08:08.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.793+0000 7f06aed55640 1 -- start start 2023-08-24T10:08:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.794+0000 7f06aed55640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06a8133700 0x7f06a8135b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.794+0000 7f06aed55640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a809a1e0 con 0x7f06a8096410 2023-08-24T10:08:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.794+0000 7f06aed55640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a8097af0 con 0x7f06a8133700 2023-08-24T10:08:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.794+0000 7f06acaf1640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f06a8096410 0x7f06a8096820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52876/0 (socket says 172.21.15.60:52876) 2023-08-24T10:08:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.794+0000 7f06acaf1640 1 -- 172.21.15.60:0/1279947012 learned_addr learned my addr 172.21.15.60:0/1279947012 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:08:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.794+0000 7f06a7fff640 1 --2- 172.21.15.60:0/1279947012 >> v2:172.21.15.60:3300/0 conn(0x7f06a8133700 0x7f06a8135b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.794+0000 7f06a7fff640 1 -- 172.21.15.60:0/1279947012 >> v1:172.21.15.60:6789/0 conn(0x7f06a8096410 legacy=0x7f06a8096820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.794+0000 7f06a7fff640 1 -- 172.21.15.60:0/1279947012 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a8096f30 con 0x7f06a8133700 2023-08-24T10:08:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.794+0000 7f06a7fff640 1 --2- 172.21.15.60:0/1279947012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06a8133700 0x7f06a8135b20 crc :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7d4282953f0437 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.795+0000 7f06a77fe640 1 -- 172.21.15.60:0/1279947012 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f069c009ce0 con 0x7f06a8133700 2023-08-24T10:08:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.795+0000 7f06a77fe640 1 -- 172.21.15.60:0/1279947012 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f069c009e40 con 0x7f06a8133700 2023-08-24T10:08:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.795+0000 7f06a77fe640 1 -- 172.21.15.60:0/1279947012 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f069c00e360 con 0x7f06a8133700 2023-08-24T10:08:08.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.795+0000 7f06aed55640 1 -- 172.21.15.60:0/1279947012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06a8133700 msgr2=0x7f06a8135b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:08.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.795+0000 7f06aed55640 1 --2- 172.21.15.60:0/1279947012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06a8133700 0x7f06a8135b20 crc :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:08.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.795+0000 7f06aed55640 1 -- 172.21.15.60:0/1279947012 shutdown_connections 2023-08-24T10:08:08.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.795+0000 7f06aed55640 1 --2- 172.21.15.60:0/1279947012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06a8133700 0x7f06a8135b20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:08.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.795+0000 7f06aed55640 1 -- 172.21.15.60:0/1279947012 >> 172.21.15.60:0/1279947012 conn(0x7f06a809e110 msgr2=0x7f06a809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:08.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.795+0000 7f06aed55640 1 -- 172.21.15.60:0/1279947012 shutdown_connections 2023-08-24T10:08:08.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.795+0000 7f06aed55640 1 -- 172.21.15.60:0/1279947012 wait complete. 2023-08-24T10:08:08.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.796+0000 7f06aed55640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:08.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.796+0000 7f06aed55640 1 Processor -- start 2023-08-24T10:08:08.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.797+0000 7f06aed55640 1 -- start start 2023-08-24T10:08:08.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.797+0000 7f06aed55640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06a8096410 0x7f06a812f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:08.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.797+0000 7f06aed55640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a806d6f0 con 0x7f06a8096410 2023-08-24T10:08:08.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.797+0000 7f06acaf1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06a8096410 0x7f06a812f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:08.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.798+0000 7f06acaf1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06a8096410 0x7f06a812f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32936/0 (socket says 172.21.15.60:32936) 2023-08-24T10:08:08.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.798+0000 7f06acaf1640 1 -- 172.21.15.60:0/1753110203 learned_addr learned my addr 172.21.15.60:0/1753110203 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:08.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.798+0000 7f06acaf1640 1 -- 172.21.15.60:0/1753110203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a806ef90 con 0x7f06a8096410 2023-08-24T10:08:08.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.798+0000 7f06acaf1640 1 --2- 172.21.15.60:0/1753110203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06a8096410 0x7f06a812f380 crc :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:08.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.798+0000 7f06a57fa640 1 -- 172.21.15.60:0/1753110203 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f06980058a0 con 0x7f06a8096410 2023-08-24T10:08:08.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.798+0000 7f06a57fa640 1 -- 172.21.15.60:0/1753110203 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0698005a00 con 0x7f06a8096410 2023-08-24T10:08:08.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.798+0000 7f06aed55640 1 -- 172.21.15.60:0/1753110203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a806d9e0 con 0x7f06a8096410 2023-08-24T10:08:08.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.798+0000 7f06a57fa640 1 -- 172.21.15.60:0/1753110203 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0698005ca0 con 0x7f06a8096410 2023-08-24T10:08:08.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.798+0000 7f06aed55640 1 -- 172.21.15.60:0/1753110203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a806d230 con 0x7f06a8096410 2023-08-24T10:08:08.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.799+0000 7f06a57fa640 1 -- 172.21.15.60:0/1753110203 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f06980070b0 con 0x7f06a8096410 2023-08-24T10:08:08.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.799+0000 7f06aed55640 1 -- 172.21.15.60:0/1753110203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a809cb20 con 0x7f06a8096410 2023-08-24T10:08:08.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.799+0000 7f06a57fa640 1 --2- 172.21.15.60:0/1753110203 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0694037130 0x7f0694039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:08.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.799+0000 7f06a57fa640 1 -- 172.21.15.60:0/1753110203 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f0698046410 con 0x7f06a8096410 2023-08-24T10:08:08.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.800+0000 7f06a7fff640 1 --2- 172.21.15.60:0/1753110203 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0694037130 0x7f0694039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:08.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.800+0000 7f06a7fff640 1 --2- 172.21.15.60:0/1753110203 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0694037130 0x7f0694039640 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:08.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:08.804+0000 7f06a57fa640 1 -- 172.21.15.60:0/1753110203 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f06a809cb20 con 0x7f06a8096410 2023-08-24T10:08:09.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.053+0000 7f06aed55640 1 -- 172.21.15.60:0/1753110203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f06a809cb20 con 0x7f06a8096410 2023-08-24T10:08:09.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.053+0000 7f06a57fa640 1 -- 172.21.15.60:0/1753110203 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f06a809cb20 con 0x7f06a8096410 2023-08-24T10:08:09.056 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:09.056 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":199,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:08:09.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.055+0000 7f06aed55640 1 -- 172.21.15.60:0/1753110203 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0694037130 msgr2=0x7f0694039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:09.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.055+0000 7f06aed55640 1 --2- 172.21.15.60:0/1753110203 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0694037130 0x7f0694039640 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:09.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.055+0000 7f06aed55640 1 -- 172.21.15.60:0/1753110203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06a8096410 msgr2=0x7f06a812f380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:09.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.055+0000 7f06aed55640 1 --2- 172.21.15.60:0/1753110203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06a8096410 0x7f06a812f380 crc :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:09.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.056+0000 7f06aed55640 1 -- 172.21.15.60:0/1753110203 shutdown_connections 2023-08-24T10:08:09.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.056+0000 7f06aed55640 1 --2- 172.21.15.60:0/1753110203 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0694037130 0x7f0694039640 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.056+0000 7f06aed55640 1 --2- 172.21.15.60:0/1753110203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06a8096410 0x7f06a812f380 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.056+0000 7f06aed55640 1 -- 172.21.15.60:0/1753110203 >> 172.21.15.60:0/1753110203 conn(0x7f06a809e110 msgr2=0x7f06a80985b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.056+0000 7f06aed55640 1 -- 172.21.15.60:0/1753110203 shutdown_connections 2023-08-24T10:08:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.056+0000 7f06aed55640 1 -- 172.21.15.60:0/1753110203 wait complete. 2023-08-24T10:08:09.068 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:08:09.069 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:09.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.175+0000 7f00e299e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:09.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.179+0000 7f00e299e640 1 Processor -- start 2023-08-24T10:08:09.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.179+0000 7f00e299e640 1 -- start start 2023-08-24T10:08:09.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.179+0000 7f00e299e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f00dc131d40 0x7f00dc134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:09.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.179+0000 7f00e299e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00dc099440 con 0x7f00dc06d300 2023-08-24T10:08:09.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.179+0000 7f00e299e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00dc09a050 con 0x7f00dc131d40 2023-08-24T10:08:09.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.179+0000 7f00db7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f00dc131d40 0x7f00dc134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:09.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.179+0000 7f00db7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f00dc131d40 0x7f00dc134160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32948/0 (socket says 172.21.15.60:32948) 2023-08-24T10:08:09.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.179+0000 7f00db7fe640 1 -- 172.21.15.60:0/3374834060 learned_addr learned my addr 172.21.15.60:0/3374834060 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:09.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.179+0000 7f00dbfff640 1 --1- 172.21.15.60:0/3374834060 >> v1:172.21.15.60:6789/0 conn(0x7f00dc06d300 0x7f00dc12bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52880/0 (socket says 172.21.15.60:52880) 2023-08-24T10:08:09.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.180+0000 7f00db7fe640 1 -- 172.21.15.60:0/3374834060 >> v1:172.21.15.60:6789/0 conn(0x7f00dc06d300 legacy=0x7f00dc12bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:09.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.180+0000 7f00db7fe640 1 -- 172.21.15.60:0/3374834060 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00dc12b9d0 con 0x7f00dc131d40 2023-08-24T10:08:09.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.180+0000 7f00db7fe640 1 --2- 172.21.15.60:0/3374834060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00dc131d40 0x7f00dc134160 crc :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a721ea9cadd89 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:09.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.180+0000 7f00daffd640 1 -- 172.21.15.60:0/3374834060 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f00cc00d580 con 0x7f00dc131d40 2023-08-24T10:08:09.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.180+0000 7f00daffd640 1 -- 172.21.15.60:0/3374834060 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f00cc00d6e0 con 0x7f00dc131d40 2023-08-24T10:08:09.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.180+0000 7f00daffd640 1 -- 172.21.15.60:0/3374834060 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f00cc00d980 con 0x7f00dc131d40 2023-08-24T10:08:09.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.181+0000 7f00e299e640 1 -- 172.21.15.60:0/3374834060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00dc131d40 msgr2=0x7f00dc134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:09.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.181+0000 7f00e299e640 1 --2- 172.21.15.60:0/3374834060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00dc131d40 0x7f00dc134160 crc :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:09.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.181+0000 7f00e299e640 1 -- 172.21.15.60:0/3374834060 shutdown_connections 2023-08-24T10:08:09.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.181+0000 7f00e299e640 1 --2- 172.21.15.60:0/3374834060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00dc131d40 0x7f00dc134160 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:09.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.181+0000 7f00e299e640 1 -- 172.21.15.60:0/3374834060 >> 172.21.15.60:0/3374834060 conn(0x7f00dc09e110 msgr2=0x7f00dc09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:09.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.181+0000 7f00e299e640 1 -- 172.21.15.60:0/3374834060 shutdown_connections 2023-08-24T10:08:09.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.181+0000 7f00e299e640 1 -- 172.21.15.60:0/3374834060 wait complete. 2023-08-24T10:08:09.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.181+0000 7f00e299e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:09.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.182+0000 7f00e299e640 1 Processor -- start 2023-08-24T10:08:09.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.182+0000 7f00e299e640 1 -- start start 2023-08-24T10:08:09.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.183+0000 7f00e299e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00dc06d300 0x7f00dc1c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.183+0000 7f00e299e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc06f4a0 con 0x7f00dc06d300 2023-08-24T10:08:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.183+0000 7f00dbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00dc06d300 0x7f00dc1c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.183+0000 7f00dbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00dc06d300 0x7f00dc1c2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32964/0 (socket says 172.21.15.60:32964) 2023-08-24T10:08:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.183+0000 7f00dbfff640 1 -- 172.21.15.60:0/4234483095 learned_addr learned my addr 172.21.15.60:0/4234483095 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:09.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.183+0000 7f00dbfff640 1 -- 172.21.15.60:0/4234483095 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00dc127580 con 0x7f00dc06d300 2023-08-24T10:08:09.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.183+0000 7f00dbfff640 1 --2- 172.21.15.60:0/4234483095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00dc06d300 0x7f00dc1c2f20 crc :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:09.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.183+0000 7f00d8ff9640 1 -- 172.21.15.60:0/4234483095 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f00d00071e0 con 0x7f00dc06d300 2023-08-24T10:08:09.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.183+0000 7f00d8ff9640 1 -- 172.21.15.60:0/4234483095 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f00d0007340 con 0x7f00dc06d300 2023-08-24T10:08:09.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.183+0000 7f00e299e640 1 -- 172.21.15.60:0/4234483095 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00dc1290d0 con 0x7f00dc06d300 2023-08-24T10:08:09.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.183+0000 7f00d8ff9640 1 -- 172.21.15.60:0/4234483095 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f00d00075e0 con 0x7f00dc06d300 2023-08-24T10:08:09.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.184+0000 7f00e299e640 1 -- 172.21.15.60:0/4234483095 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00dc06fe70 con 0x7f00dc06d300 2023-08-24T10:08:09.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.185+0000 7f00d8ff9640 1 -- 172.21.15.60:0/4234483095 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f00d00057e0 con 0x7f00dc06d300 2023-08-24T10:08:09.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.185+0000 7f00e299e640 1 -- 172.21.15.60:0/4234483095 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00dc061800 con 0x7f00dc06d300 2023-08-24T10:08:09.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.185+0000 7f00d8ff9640 1 --2- 172.21.15.60:0/4234483095 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f00c8037130 0x7f00c8039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:09.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.185+0000 7f00d8ff9640 1 -- 172.21.15.60:0/4234483095 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f00d0046230 con 0x7f00dc06d300 2023-08-24T10:08:09.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.185+0000 7f00db7fe640 1 --2- 172.21.15.60:0/4234483095 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f00c8037130 0x7f00c8039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:09.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.185+0000 7f00db7fe640 1 --2- 172.21.15.60:0/4234483095 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f00c8037130 0x7f00c8039640 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:09.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.189+0000 7f00d8ff9640 1 -- 172.21.15.60:0/4234483095 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f00d000d330 con 0x7f00dc06d300 2023-08-24T10:08:09.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.359+0000 7f00e299e640 1 -- 172.21.15.60:0/4234483095 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f00dc12a310 con 0x7f00c8037130 2023-08-24T10:08:09.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.361+0000 7f00d8ff9640 1 -- 172.21.15.60:0/4234483095 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f00cc00d850 con 0x7f00c8037130 2023-08-24T10:08:09.363 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:09.364 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:09.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.363+0000 7f00e299e640 1 -- 172.21.15.60:0/4234483095 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f00c8037130 msgr2=0x7f00c8039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:09.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.363+0000 7f00e299e640 1 --2- 172.21.15.60:0/4234483095 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f00c8037130 0x7f00c8039640 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:09.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.363+0000 7f00e299e640 1 -- 172.21.15.60:0/4234483095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00dc06d300 msgr2=0x7f00dc1c2f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:09.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.363+0000 7f00e299e640 1 --2- 172.21.15.60:0/4234483095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00dc06d300 0x7f00dc1c2f20 crc :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:09.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.363+0000 7f00e299e640 1 -- 172.21.15.60:0/4234483095 shutdown_connections 2023-08-24T10:08:09.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.363+0000 7f00e299e640 1 --2- 172.21.15.60:0/4234483095 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f00c8037130 0x7f00c8039640 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:09.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.363+0000 7f00e299e640 1 --2- 172.21.15.60:0/4234483095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00dc06d300 0x7f00dc1c2f20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:09.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.363+0000 7f00e299e640 1 -- 172.21.15.60:0/4234483095 >> 172.21.15.60:0/4234483095 conn(0x7f00dc09e110 msgr2=0x7f00dc12ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:09.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.363+0000 7f00e299e640 1 -- 172.21.15.60:0/4234483095 shutdown_connections 2023-08-24T10:08:09.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:09.363+0000 7f00e299e640 1 -- 172.21.15.60:0/4234483095 wait complete. 2023-08-24T10:08:09.376 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2023-08-24T10:08:07.531322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:09.583 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~182s 2023-08-24T10:08:09.583 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~182s 2023-08-24T10:08:09.583 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~182s 2023-08-24T10:08:09.583 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~182s 2023-08-24T10:08:12.377 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:12.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.498+0000 7eff8b630640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:12.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.502+0000 7eff8b630640 1 Processor -- start 2023-08-24T10:08:12.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.502+0000 7eff8b630640 1 -- start start 2023-08-24T10:08:12.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.503+0000 7eff8b630640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7eff84070110 0x7eff841356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:12.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.503+0000 7eff8b630640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff84096f10 con 0x7eff84071e40 2023-08-24T10:08:12.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.503+0000 7eff8b630640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff840977f0 con 0x7eff84070110 2023-08-24T10:08:12.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.503+0000 7eff88bcb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7eff84070110 0x7eff841356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:12.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.503+0000 7eff88bcb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7eff84070110 0x7eff841356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32978/0 (socket says 172.21.15.60:32978) 2023-08-24T10:08:12.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.503+0000 7eff88bcb640 1 -- 172.21.15.60:0/713881744 learned_addr learned my addr 172.21.15.60:0/713881744 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:12.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.503+0000 7eff893cc640 1 --1- 172.21.15.60:0/713881744 >> v1:172.21.15.60:6789/0 conn(0x7eff84071e40 0x7eff84096800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52882/0 (socket says 172.21.15.60:52882) 2023-08-24T10:08:12.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.503+0000 7eff88bcb640 1 -- 172.21.15.60:0/713881744 >> v1:172.21.15.60:6789/0 conn(0x7eff84071e40 legacy=0x7eff84096800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:12.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.503+0000 7eff88bcb640 1 -- 172.21.15.60:0/713881744 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8409a280 con 0x7eff84070110 2023-08-24T10:08:12.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.503+0000 7eff88bcb640 1 --2- 172.21.15.60:0/713881744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff84070110 0x7eff841356a0 crc :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90be4a7f374cb27 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:12.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.504+0000 7eff7bfff640 1 -- 172.21.15.60:0/713881744 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7eff74009dd0 con 0x7eff84070110 2023-08-24T10:08:12.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.504+0000 7eff7bfff640 1 -- 172.21.15.60:0/713881744 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7eff7400e230 con 0x7eff84070110 2023-08-24T10:08:12.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.504+0000 7eff7bfff640 1 -- 172.21.15.60:0/713881744 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7eff7400e4d0 con 0x7eff84070110 2023-08-24T10:08:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.504+0000 7eff8b630640 1 -- 172.21.15.60:0/713881744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff84070110 msgr2=0x7eff841356a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.504+0000 7eff8b630640 1 --2- 172.21.15.60:0/713881744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff84070110 0x7eff841356a0 crc :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.504+0000 7eff8b630640 1 -- 172.21.15.60:0/713881744 shutdown_connections 2023-08-24T10:08:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.504+0000 7eff8b630640 1 --2- 172.21.15.60:0/713881744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff84070110 0x7eff841356a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.504+0000 7eff8b630640 1 -- 172.21.15.60:0/713881744 >> 172.21.15.60:0/713881744 conn(0x7eff8409e110 msgr2=0x7eff8409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.504+0000 7eff8b630640 1 -- 172.21.15.60:0/713881744 shutdown_connections 2023-08-24T10:08:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.504+0000 7eff8b630640 1 -- 172.21.15.60:0/713881744 wait complete. 2023-08-24T10:08:12.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.505+0000 7eff8b630640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:12.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.506+0000 7eff8b630640 1 Processor -- start 2023-08-24T10:08:12.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.506+0000 7eff8b630640 1 -- start start 2023-08-24T10:08:12.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.506+0000 7eff8b630640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff84070110 0x7eff84093460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:12.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.506+0000 7eff8b630640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8408ff10 con 0x7eff84070110 2023-08-24T10:08:12.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.506+0000 7eff893cc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff84070110 0x7eff84093460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:12.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.506+0000 7eff893cc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff84070110 0x7eff84093460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32980/0 (socket says 172.21.15.60:32980) 2023-08-24T10:08:12.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.506+0000 7eff893cc640 1 -- 172.21.15.60:0/1445388755 learned_addr learned my addr 172.21.15.60:0/1445388755 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:12.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.507+0000 7eff893cc640 1 -- 172.21.15.60:0/1445388755 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff84090680 con 0x7eff84070110 2023-08-24T10:08:12.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.507+0000 7eff893cc640 1 --2- 172.21.15.60:0/1445388755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff84070110 0x7eff84093460 crc :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:12.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.507+0000 7eff79ffb640 1 -- 172.21.15.60:0/1445388755 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7eff800071e0 con 0x7eff84070110 2023-08-24T10:08:12.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.507+0000 7eff79ffb640 1 -- 172.21.15.60:0/1445388755 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7eff80007340 con 0x7eff84070110 2023-08-24T10:08:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.507+0000 7eff79ffb640 1 -- 172.21.15.60:0/1445388755 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7eff800075e0 con 0x7eff84070110 2023-08-24T10:08:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.507+0000 7eff8b630640 1 -- 172.21.15.60:0/1445388755 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff84092e80 con 0x7eff84070110 2023-08-24T10:08:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.507+0000 7eff8b630640 1 -- 172.21.15.60:0/1445388755 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff84092b70 con 0x7eff84070110 2023-08-24T10:08:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.508+0000 7eff79ffb640 1 -- 172.21.15.60:0/1445388755 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7eff800057e0 con 0x7eff84070110 2023-08-24T10:08:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.508+0000 7eff79ffb640 1 --2- 172.21.15.60:0/1445388755 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7eff68037130 0x7eff68039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.508+0000 7eff79ffb640 1 -- 172.21.15.60:0/1445388755 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7eff800462f0 con 0x7eff84070110 2023-08-24T10:08:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.508+0000 7eff8b630640 1 -- 172.21.15.60:0/1445388755 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff84061800 con 0x7eff84070110 2023-08-24T10:08:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.509+0000 7eff88bcb640 1 --2- 172.21.15.60:0/1445388755 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7eff68037130 0x7eff68039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.509+0000 7eff88bcb640 1 --2- 172.21.15.60:0/1445388755 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7eff68037130 0x7eff68039640 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.513+0000 7eff79ffb640 1 -- 172.21.15.60:0/1445388755 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7eff8000d740 con 0x7eff84070110 2023-08-24T10:08:12.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.688+0000 7eff8b630640 1 -- 172.21.15.60:0/1445388755 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7eff840932f0 con 0x7eff68037130 2023-08-24T10:08:12.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.689+0000 7eff79ffb640 1 -- 172.21.15.60:0/1445388755 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7eff840932f0 con 0x7eff68037130 2023-08-24T10:08:12.690 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:12.691 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:12.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.691+0000 7eff8b630640 1 -- 172.21.15.60:0/1445388755 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7eff68037130 msgr2=0x7eff68039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:12.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.691+0000 7eff8b630640 1 --2- 172.21.15.60:0/1445388755 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7eff68037130 0x7eff68039640 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:12.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.691+0000 7eff8b630640 1 -- 172.21.15.60:0/1445388755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff84070110 msgr2=0x7eff84093460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:12.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.691+0000 7eff8b630640 1 --2- 172.21.15.60:0/1445388755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff84070110 0x7eff84093460 crc :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:12.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.691+0000 7eff8b630640 1 -- 172.21.15.60:0/1445388755 shutdown_connections 2023-08-24T10:08:12.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.691+0000 7eff8b630640 1 --2- 172.21.15.60:0/1445388755 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7eff68037130 0x7eff68039640 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:12.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.691+0000 7eff8b630640 1 --2- 172.21.15.60:0/1445388755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff84070110 0x7eff84093460 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:12.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.691+0000 7eff8b630640 1 -- 172.21.15.60:0/1445388755 >> 172.21.15.60:0/1445388755 conn(0x7eff8409e110 msgr2=0x7eff84098490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:12.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.691+0000 7eff8b630640 1 -- 172.21.15.60:0/1445388755 shutdown_connections 2023-08-24T10:08:12.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.691+0000 7eff8b630640 1 -- 172.21.15.60:0/1445388755 wait complete. 2023-08-24T10:08:12.703 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2023-08-24T10:08:11.532156+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:12.705 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:12.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.812+0000 7f0c7db5e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:12.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.816+0000 7f0c7db5e640 1 Processor -- start 2023-08-24T10:08:12.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.816+0000 7f0c7db5e640 1 -- start start 2023-08-24T10:08:12.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.817+0000 7f0c7db5e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0c7806d300 0x7f0c7812bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:12.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.817+0000 7f0c7db5e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c78099440 con 0x7f0c7812c1c0 2023-08-24T10:08:12.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.817+0000 7f0c7db5e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c7809a050 con 0x7f0c7806d300 2023-08-24T10:08:12.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.817+0000 7f0c777fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0c7806d300 0x7f0c7812bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:12.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.817+0000 7f0c777fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0c7806d300 0x7f0c7812bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32982/0 (socket says 172.21.15.60:32982) 2023-08-24T10:08:12.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.817+0000 7f0c777fe640 1 -- 172.21.15.60:0/3876982424 learned_addr learned my addr 172.21.15.60:0/3876982424 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:12.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.817+0000 7f0c76ffd640 1 --1- 172.21.15.60:0/3876982424 >> v1:172.21.15.60:6789/0 conn(0x7f0c7812c1c0 0x7f0c7812ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52892/0 (socket says 172.21.15.60:52892) 2023-08-24T10:08:12.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.817+0000 7f0c777fe640 1 -- 172.21.15.60:0/3876982424 >> v1:172.21.15.60:6789/0 conn(0x7f0c7812c1c0 legacy=0x7f0c7812ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:12.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.817+0000 7f0c777fe640 1 -- 172.21.15.60:0/3876982424 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7812b9d0 con 0x7f0c7806d300 2023-08-24T10:08:12.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.818+0000 7f0c777fe640 1 --2- 172.21.15.60:0/3876982424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c7806d300 0x7f0c7812bc80 crc :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6c0b9f812e5969 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:12.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.818+0000 7f0c767fc640 1 -- 172.21.15.60:0/3876982424 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0c68009dd0 con 0x7f0c7806d300 2023-08-24T10:08:12.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.818+0000 7f0c767fc640 1 -- 172.21.15.60:0/3876982424 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0c6800e230 con 0x7f0c7806d300 2023-08-24T10:08:12.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.818+0000 7f0c767fc640 1 -- 172.21.15.60:0/3876982424 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0c6800e4d0 con 0x7f0c7806d300 2023-08-24T10:08:12.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.818+0000 7f0c7db5e640 1 -- 172.21.15.60:0/3876982424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c7806d300 msgr2=0x7f0c7812bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:12.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.818+0000 7f0c7db5e640 1 --2- 172.21.15.60:0/3876982424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c7806d300 0x7f0c7812bc80 crc :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:12.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.818+0000 7f0c7db5e640 1 -- 172.21.15.60:0/3876982424 shutdown_connections 2023-08-24T10:08:12.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.818+0000 7f0c7db5e640 1 --2- 172.21.15.60:0/3876982424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c7806d300 0x7f0c7812bc80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:12.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.818+0000 7f0c7db5e640 1 -- 172.21.15.60:0/3876982424 >> 172.21.15.60:0/3876982424 conn(0x7f0c7809e110 msgr2=0x7f0c7809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:12.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.818+0000 7f0c7db5e640 1 -- 172.21.15.60:0/3876982424 shutdown_connections 2023-08-24T10:08:12.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.818+0000 7f0c7db5e640 1 -- 172.21.15.60:0/3876982424 wait complete. 2023-08-24T10:08:12.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.819+0000 7f0c7db5e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:12.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.820+0000 7f0c7db5e640 1 Processor -- start 2023-08-24T10:08:12.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.820+0000 7f0c7db5e640 1 -- start start 2023-08-24T10:08:12.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.820+0000 7f0c7db5e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c7806d300 0x7f0c7806eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:12.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.821+0000 7f0c7db5e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7806ea70 con 0x7f0c7806d300 2023-08-24T10:08:12.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.821+0000 7f0c777fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c7806d300 0x7f0c7806eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:12.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.821+0000 7f0c777fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c7806d300 0x7f0c7806eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32990/0 (socket says 172.21.15.60:32990) 2023-08-24T10:08:12.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.821+0000 7f0c777fe640 1 -- 172.21.15.60:0/2785116832 learned_addr learned my addr 172.21.15.60:0/2785116832 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:12.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.821+0000 7f0c777fe640 1 -- 172.21.15.60:0/2785116832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7806ffe0 con 0x7f0c7806d300 2023-08-24T10:08:12.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.821+0000 7f0c777fe640 1 --2- 172.21.15.60:0/2785116832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c7806d300 0x7f0c7806eea0 crc :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:12.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.821+0000 7f0c5ffff640 1 -- 172.21.15.60:0/2785116832 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0c68014540 con 0x7f0c7806d300 2023-08-24T10:08:12.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.821+0000 7f0c5ffff640 1 -- 172.21.15.60:0/2785116832 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0c680146a0 con 0x7f0c7806d300 2023-08-24T10:08:12.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.821+0000 7f0c7db5e640 1 -- 172.21.15.60:0/2785116832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c78071000 con 0x7f0c7806d300 2023-08-24T10:08:12.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.821+0000 7f0c5ffff640 1 -- 172.21.15.60:0/2785116832 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0c68014940 con 0x7f0c7806d300 2023-08-24T10:08:12.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.821+0000 7f0c7db5e640 1 -- 172.21.15.60:0/2785116832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7806e760 con 0x7f0c7806d300 2023-08-24T10:08:12.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.822+0000 7f0c5ffff640 1 -- 172.21.15.60:0/2785116832 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f0c68014af0 con 0x7f0c7806d300 2023-08-24T10:08:12.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.822+0000 7f0c7db5e640 1 -- 172.21.15.60:0/2785116832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7809c110 con 0x7f0c7806d300 2023-08-24T10:08:12.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.822+0000 7f0c5ffff640 1 --2- 172.21.15.60:0/2785116832 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0c64037130 0x7f0c64039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:12.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.822+0000 7f0c5ffff640 1 -- 172.21.15.60:0/2785116832 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f0c68019040 con 0x7f0c7806d300 2023-08-24T10:08:12.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.822+0000 7f0c76ffd640 1 --2- 172.21.15.60:0/2785116832 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0c64037130 0x7f0c64039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:12.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.823+0000 7f0c76ffd640 1 --2- 172.21.15.60:0/2785116832 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0c64037130 0x7f0c64039640 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:12.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.827+0000 7f0c5ffff640 1 -- 172.21.15.60:0/2785116832 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f0c7809c110 con 0x7f0c7806d300 2023-08-24T10:08:13.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:12.998+0000 7f0c7db5e640 1 -- 172.21.15.60:0/2785116832 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c7806d710 con 0x7f0c64037130 2023-08-24T10:08:13.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.000+0000 7f0c5ffff640 1 -- 172.21.15.60:0/2785116832 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26401 (crc 0 0 0) 0x7f0c6c001210 con 0x7f0c64037130 2023-08-24T10:08:13.001 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:13.001 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:13.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.001+0000 7f0c7db5e640 1 -- 172.21.15.60:0/2785116832 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0c64037130 msgr2=0x7f0c64039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:13.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.001+0000 7f0c7db5e640 1 --2- 172.21.15.60:0/2785116832 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0c64037130 0x7f0c64039640 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:13.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.001+0000 7f0c7db5e640 1 -- 172.21.15.60:0/2785116832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c7806d300 msgr2=0x7f0c7806eea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:13.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.001+0000 7f0c7db5e640 1 --2- 172.21.15.60:0/2785116832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c7806d300 0x7f0c7806eea0 crc :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:13.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.001+0000 7f0c7db5e640 1 -- 172.21.15.60:0/2785116832 shutdown_connections 2023-08-24T10:08:13.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.001+0000 7f0c7db5e640 1 --2- 172.21.15.60:0/2785116832 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0c64037130 0x7f0c64039640 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:13.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.001+0000 7f0c7db5e640 1 --2- 172.21.15.60:0/2785116832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c7806d300 0x7f0c7806eea0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:13.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.001+0000 7f0c7db5e640 1 -- 172.21.15.60:0/2785116832 >> 172.21.15.60:0/2785116832 conn(0x7f0c7809e110 msgr2=0x7f0c7809ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:13.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.001+0000 7f0c7db5e640 1 -- 172.21.15.60:0/2785116832 shutdown_connections 2023-08-24T10:08:13.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.001+0000 7f0c7db5e640 1 -- 172.21.15.60:0/2785116832 wait complete. 2023-08-24T10:08:13.015 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2023-08-24T10:08:11.532156+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:13.017 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:08:13.017 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:13.017 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:08:13.018 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:13.018 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:08:13.018 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:13.018 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:08:13.019 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:13.019 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:08:13.019 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:13.019 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:08:13.020 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:13.020 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:08:13.020 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:13.021 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:08:13.021 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:13.021 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:08:13.021 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:13.022 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:08:13.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.128+0000 7fd49af9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:13.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.131+0000 7fd49af9e640 1 Processor -- start 2023-08-24T10:08:13.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.131+0000 7fd49af9e640 1 -- start start 2023-08-24T10:08:13.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.132+0000 7fd49af9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd49412ece0 0x7fd49412f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:13.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.132+0000 7fd49af9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd49409a650 con 0x7fd49412f630 2023-08-24T10:08:13.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.132+0000 7fd49af9e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd49406da00 con 0x7fd49412ece0 2023-08-24T10:08:13.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.132+0000 7fd498d3a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd49412ece0 0x7fd49412f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:13.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.132+0000 7fd498d3a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd49412ece0 0x7fd49412f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33004/0 (socket says 172.21.15.60:33004) 2023-08-24T10:08:13.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.132+0000 7fd498d3a640 1 -- 172.21.15.60:0/2669674137 learned_addr learned my addr 172.21.15.60:0/2669674137 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:13.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.132+0000 7fd498d3a640 1 -- 172.21.15.60:0/2669674137 >> v1:172.21.15.60:6789/0 conn(0x7fd49412f630 legacy=0x7fd494097fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:13.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.132+0000 7fd498d3a640 1 -- 172.21.15.60:0/2669674137 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49412ea10 con 0x7fd49412ece0 2023-08-24T10:08:13.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.133+0000 7fd498d3a640 1 --2- 172.21.15.60:0/2669674137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd49412ece0 0x7fd49412f0f0 crc :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94de988af0a0bfe server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:13.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.133+0000 7fd48b7fe640 1 -- 172.21.15.60:0/2669674137 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd490009dd0 con 0x7fd49412ece0 2023-08-24T10:08:13.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.133+0000 7fd48b7fe640 1 -- 172.21.15.60:0/2669674137 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd49000e230 con 0x7fd49412ece0 2023-08-24T10:08:13.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.133+0000 7fd48b7fe640 1 -- 172.21.15.60:0/2669674137 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd49000e4d0 con 0x7fd49412ece0 2023-08-24T10:08:13.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.133+0000 7fd49af9e640 1 -- 172.21.15.60:0/2669674137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd49412ece0 msgr2=0x7fd49412f0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:13.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.133+0000 7fd49af9e640 1 --2- 172.21.15.60:0/2669674137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd49412ece0 0x7fd49412f0f0 crc :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:13.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.133+0000 7fd49af9e640 1 -- 172.21.15.60:0/2669674137 shutdown_connections 2023-08-24T10:08:13.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.133+0000 7fd49af9e640 1 --2- 172.21.15.60:0/2669674137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd49412ece0 0x7fd49412f0f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:13.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.133+0000 7fd49af9e640 1 -- 172.21.15.60:0/2669674137 >> 172.21.15.60:0/2669674137 conn(0x7fd49409e110 msgr2=0x7fd49409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:13.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.133+0000 7fd49af9e640 1 -- 172.21.15.60:0/2669674137 shutdown_connections 2023-08-24T10:08:13.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.133+0000 7fd49af9e640 1 -- 172.21.15.60:0/2669674137 wait complete. 2023-08-24T10:08:13.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.134+0000 7fd49af9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:13.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.135+0000 7fd49af9e640 1 Processor -- start 2023-08-24T10:08:13.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.135+0000 7fd49af9e640 1 -- start start 2023-08-24T10:08:13.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.135+0000 7fd49af9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd49412ece0 0x7fd494133b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:13.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.135+0000 7fd49af9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49406b200 con 0x7fd49412ece0 2023-08-24T10:08:13.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.135+0000 7fd498d3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd49412ece0 0x7fd494133b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:13.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.135+0000 7fd498d3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd49412ece0 0x7fd494133b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33020/0 (socket says 172.21.15.60:33020) 2023-08-24T10:08:13.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.135+0000 7fd498d3a640 1 -- 172.21.15.60:0/331560789 learned_addr learned my addr 172.21.15.60:0/331560789 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:13.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.136+0000 7fd498d3a640 1 -- 172.21.15.60:0/331560789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd494133840 con 0x7fd49412ece0 2023-08-24T10:08:13.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.136+0000 7fd498d3a640 1 --2- 172.21.15.60:0/331560789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd49412ece0 0x7fd494133b50 crc :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:13.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.136+0000 7fd4897fa640 1 -- 172.21.15.60:0/331560789 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd490018040 con 0x7fd49412ece0 2023-08-24T10:08:13.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.136+0000 7fd4897fa640 1 -- 172.21.15.60:0/331560789 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd490012650 con 0x7fd49412ece0 2023-08-24T10:08:13.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.136+0000 7fd4897fa640 1 -- 172.21.15.60:0/331560789 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd4900128f0 con 0x7fd49412ece0 2023-08-24T10:08:13.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.136+0000 7fd49af9e640 1 -- 172.21.15.60:0/331560789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd494126990 con 0x7fd49412ece0 2023-08-24T10:08:13.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.136+0000 7fd49af9e640 1 -- 172.21.15.60:0/331560789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd494125f20 con 0x7fd49412ece0 2023-08-24T10:08:13.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.137+0000 7fd4897fa640 1 -- 172.21.15.60:0/331560789 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fd490012af0 con 0x7fd49412ece0 2023-08-24T10:08:13.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.137+0000 7fd49af9e640 1 -- 172.21.15.60:0/331560789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd494134f10 con 0x7fd49412ece0 2023-08-24T10:08:13.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.137+0000 7fd4897fa640 1 --2- 172.21.15.60:0/331560789 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd480037130 0x7fd480039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:13.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.137+0000 7fd4897fa640 1 -- 172.21.15.60:0/331560789 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fd490049430 con 0x7fd49412ece0 2023-08-24T10:08:13.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.137+0000 7fd48bfff640 1 --2- 172.21.15.60:0/331560789 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd480037130 0x7fd480039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:13.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.138+0000 7fd48bfff640 1 --2- 172.21.15.60:0/331560789 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd480037130 0x7fd480039640 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:13.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.142+0000 7fd4897fa640 1 -- 172.21.15.60:0/331560789 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fd4900176c0 con 0x7fd49412ece0 2023-08-24T10:08:13.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.390+0000 7fd49af9e640 1 -- 172.21.15.60:0/331560789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd494061800 con 0x7fd49412ece0 2023-08-24T10:08:13.391 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.390+0000 7fd4897fa640 1 -- 172.21.15.60:0/331560789 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7fd490019e00 con 0x7fd49412ece0 2023-08-24T10:08:13.392 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:13.392 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":204,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:08:13.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.392+0000 7fd49af9e640 1 -- 172.21.15.60:0/331560789 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd480037130 msgr2=0x7fd480039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:13.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.392+0000 7fd49af9e640 1 --2- 172.21.15.60:0/331560789 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd480037130 0x7fd480039640 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:13.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.392+0000 7fd49af9e640 1 -- 172.21.15.60:0/331560789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd49412ece0 msgr2=0x7fd494133b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:13.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.392+0000 7fd49af9e640 1 --2- 172.21.15.60:0/331560789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd49412ece0 0x7fd494133b50 crc :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:13.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.392+0000 7fd49af9e640 1 -- 172.21.15.60:0/331560789 shutdown_connections 2023-08-24T10:08:13.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.392+0000 7fd49af9e640 1 --2- 172.21.15.60:0/331560789 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd480037130 0x7fd480039640 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:13.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.392+0000 7fd49af9e640 1 --2- 172.21.15.60:0/331560789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd49412ece0 0x7fd494133b50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:13.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.392+0000 7fd49af9e640 1 -- 172.21.15.60:0/331560789 >> 172.21.15.60:0/331560789 conn(0x7fd49409e110 msgr2=0x7fd494097710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:13.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.392+0000 7fd49af9e640 1 -- 172.21.15.60:0/331560789 shutdown_connections 2023-08-24T10:08:13.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.393+0000 7fd49af9e640 1 -- 172.21.15.60:0/331560789 wait complete. 2023-08-24T10:08:13.404 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:08:13.405 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:13.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.511+0000 7f368af9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:13.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.515+0000 7f368af9e640 1 Processor -- start 2023-08-24T10:08:13.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.515+0000 7f368af9e640 1 -- start start 2023-08-24T10:08:13.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.516+0000 7f368af9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3684131df0 0x7f3684134210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:13.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.516+0000 7f368af9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368412ea80 con 0x7f368412f030 2023-08-24T10:08:13.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.516+0000 7f368af9e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368406d880 con 0x7f3684131df0 2023-08-24T10:08:13.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.516+0000 7f3683fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3684131df0 0x7f3684134210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:13.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.516+0000 7f3683fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3684131df0 0x7f3684134210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33032/0 (socket says 172.21.15.60:33032) 2023-08-24T10:08:13.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.516+0000 7f3688d3a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f368412f030 0x7f368412f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52910/0 (socket says 172.21.15.60:52910) 2023-08-24T10:08:13.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.516+0000 7f3683fff640 1 -- 172.21.15.60:0/3662948667 learned_addr learned my addr 172.21.15.60:0/3662948667 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:13.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.516+0000 7f3683fff640 1 -- 172.21.15.60:0/3662948667 >> v1:172.21.15.60:6789/0 conn(0x7f368412f030 legacy=0x7f368412f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:13.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.516+0000 7f3683fff640 1 -- 172.21.15.60:0/3662948667 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368406cf40 con 0x7f3684131df0 2023-08-24T10:08:13.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.516+0000 7f3683fff640 1 --2- 172.21.15.60:0/3662948667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3684131df0 0x7f3684134210 crc :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bf62f52ef7fb77 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:13.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.516+0000 7f36837fe640 1 -- 172.21.15.60:0/3662948667 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3678009dd0 con 0x7f3684131df0 2023-08-24T10:08:13.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.516+0000 7f36837fe640 1 -- 172.21.15.60:0/3662948667 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f367800e230 con 0x7f3684131df0 2023-08-24T10:08:13.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.517+0000 7f36837fe640 1 -- 172.21.15.60:0/3662948667 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f367800e4d0 con 0x7f3684131df0 2023-08-24T10:08:13.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.517+0000 7f368af9e640 1 -- 172.21.15.60:0/3662948667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3684131df0 msgr2=0x7f3684134210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:13.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.517+0000 7f368af9e640 1 --2- 172.21.15.60:0/3662948667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3684131df0 0x7f3684134210 crc :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:13.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.517+0000 7f368af9e640 1 -- 172.21.15.60:0/3662948667 shutdown_connections 2023-08-24T10:08:13.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.517+0000 7f368af9e640 1 --2- 172.21.15.60:0/3662948667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3684131df0 0x7f3684134210 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:13.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.517+0000 7f368af9e640 1 -- 172.21.15.60:0/3662948667 >> 172.21.15.60:0/3662948667 conn(0x7f368409e0e0 msgr2=0x7f368409e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:13.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.517+0000 7f368af9e640 1 -- 172.21.15.60:0/3662948667 shutdown_connections 2023-08-24T10:08:13.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.517+0000 7f368af9e640 1 -- 172.21.15.60:0/3662948667 wait complete. 2023-08-24T10:08:13.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.517+0000 7f368af9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:13.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.518+0000 7f368af9e640 1 Processor -- start 2023-08-24T10:08:13.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.518+0000 7f368af9e640 1 -- start start 2023-08-24T10:08:13.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.519+0000 7f368af9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f368412f030 0x7f36841275e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:13.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.519+0000 7f368af9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368406fee0 con 0x7f368412f030 2023-08-24T10:08:13.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.519+0000 7f3688d3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f368412f030 0x7f36841275e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:13.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.519+0000 7f3688d3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f368412f030 0x7f36841275e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33046/0 (socket says 172.21.15.60:33046) 2023-08-24T10:08:13.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.519+0000 7f3688d3a640 1 -- 172.21.15.60:0/860591037 learned_addr learned my addr 172.21.15.60:0/860591037 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:13.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.520+0000 7f3688d3a640 1 -- 172.21.15.60:0/860591037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3684071ac0 con 0x7f368412f030 2023-08-24T10:08:13.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.520+0000 7f3688d3a640 1 --2- 172.21.15.60:0/860591037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f368412f030 0x7f36841275e0 crc :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:13.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.520+0000 7f36817fa640 1 -- 172.21.15.60:0/860591037 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f36740071e0 con 0x7f368412f030 2023-08-24T10:08:13.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.520+0000 7f36817fa640 1 -- 172.21.15.60:0/860591037 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3674007340 con 0x7f368412f030 2023-08-24T10:08:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.520+0000 7f36817fa640 1 -- 172.21.15.60:0/860591037 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f36740075e0 con 0x7f368412f030 2023-08-24T10:08:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.520+0000 7f368af9e640 1 -- 172.21.15.60:0/860591037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368406afc0 con 0x7f368412f030 2023-08-24T10:08:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.520+0000 7f368af9e640 1 -- 172.21.15.60:0/860591037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3684125730 con 0x7f368412f030 2023-08-24T10:08:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.521+0000 7f36817fa640 1 -- 172.21.15.60:0/860591037 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f36740057e0 con 0x7f368412f030 2023-08-24T10:08:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.521+0000 7f368af9e640 1 -- 172.21.15.60:0/860591037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3684061800 con 0x7f368412f030 2023-08-24T10:08:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.521+0000 7f36817fa640 1 --2- 172.21.15.60:0/860591037 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3670037130 0x7f3670039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.521+0000 7f36817fa640 1 -- 172.21.15.60:0/860591037 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f36740462a0 con 0x7f368412f030 2023-08-24T10:08:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.521+0000 7f3683fff640 1 --2- 172.21.15.60:0/860591037 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3670037130 0x7f3670039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:13.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.522+0000 7f3683fff640 1 --2- 172.21.15.60:0/860591037 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3670037130 0x7f3670039640 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:13.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.526+0000 7f36817fa640 1 -- 172.21.15.60:0/860591037 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f367400d330 con 0x7f368412f030 2023-08-24T10:08:13.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.698+0000 7f368af9e640 1 -- 172.21.15.60:0/860591037 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3684126060 con 0x7f3670037130 2023-08-24T10:08:13.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.699+0000 7f36817fa640 1 -- 172.21.15.60:0/860591037 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f3678009d20 con 0x7f3670037130 2023-08-24T10:08:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:13.703 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:13.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.701+0000 7f368af9e640 1 -- 172.21.15.60:0/860591037 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3670037130 msgr2=0x7f3670039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:13.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.701+0000 7f368af9e640 1 --2- 172.21.15.60:0/860591037 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3670037130 0x7f3670039640 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:13.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.701+0000 7f368af9e640 1 -- 172.21.15.60:0/860591037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f368412f030 msgr2=0x7f36841275e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:13.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.701+0000 7f368af9e640 1 --2- 172.21.15.60:0/860591037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f368412f030 0x7f36841275e0 crc :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:13.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.701+0000 7f368af9e640 1 -- 172.21.15.60:0/860591037 shutdown_connections 2023-08-24T10:08:13.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.701+0000 7f368af9e640 1 --2- 172.21.15.60:0/860591037 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3670037130 0x7f3670039640 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:13.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.701+0000 7f368af9e640 1 --2- 172.21.15.60:0/860591037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f368412f030 0x7f36841275e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:13.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.701+0000 7f368af9e640 1 -- 172.21.15.60:0/860591037 >> 172.21.15.60:0/860591037 conn(0x7f368409e0e0 msgr2=0x7f3684130af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:13.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.701+0000 7f368af9e640 1 -- 172.21.15.60:0/860591037 shutdown_connections 2023-08-24T10:08:13.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:13.701+0000 7f368af9e640 1 -- 172.21.15.60:0/860591037 wait complete. 2023-08-24T10:08:13.713 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2023-08-24T10:08:13.532449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:14.786 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~187s 2023-08-24T10:08:14.786 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~187s 2023-08-24T10:08:14.786 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~187s 2023-08-24T10:08:14.786 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~187s 2023-08-24T10:08:16.714 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:16.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.834+0000 7f9c7bc6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:16.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.838+0000 7f9c7bc6c640 1 Processor -- start 2023-08-24T10:08:16.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.839+0000 7f9c7bc6c640 1 -- start start 2023-08-24T10:08:16.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.839+0000 7f9c7bc6c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c74132630 0x7f9c74134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:16.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.839+0000 7f9c7bc6c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c7406e320 con 0x7f9c74096860 2023-08-24T10:08:16.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.839+0000 7f9c7bc6c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c7406db70 con 0x7f9c74132630 2023-08-24T10:08:16.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.840+0000 7f9c79207640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c74132630 0x7f9c74134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:16.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.840+0000 7f9c79207640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c74132630 0x7f9c74134a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48006/0 (socket says 172.21.15.60:48006) 2023-08-24T10:08:16.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.840+0000 7f9c79207640 1 -- 172.21.15.60:0/3146685148 learned_addr learned my addr 172.21.15.60:0/3146685148 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:16.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.840+0000 7f9c79207640 1 -- 172.21.15.60:0/3146685148 >> v1:172.21.15.60:6789/0 conn(0x7f9c74096860 legacy=0x7f9c74096c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:16.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.840+0000 7f9c79207640 1 -- 172.21.15.60:0/3146685148 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c7406cef0 con 0x7f9c74132630 2023-08-24T10:08:16.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.840+0000 7f9c79207640 1 --2- 172.21.15.60:0/3146685148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c74132630 0x7f9c74134a50 crc :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7dad99f22a356fb server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:16.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.840+0000 7f9c78a06640 1 -- 172.21.15.60:0/3146685148 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9c64009dd0 con 0x7f9c74132630 2023-08-24T10:08:16.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.840+0000 7f9c78a06640 1 -- 172.21.15.60:0/3146685148 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f9c6400e230 con 0x7f9c74132630 2023-08-24T10:08:16.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.840+0000 7f9c78a06640 1 -- 172.21.15.60:0/3146685148 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9c6400e4d0 con 0x7f9c74132630 2023-08-24T10:08:16.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.841+0000 7f9c7bc6c640 1 -- 172.21.15.60:0/3146685148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c74132630 msgr2=0x7f9c74134a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:16.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.841+0000 7f9c7bc6c640 1 --2- 172.21.15.60:0/3146685148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c74132630 0x7f9c74134a50 crc :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:16.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.841+0000 7f9c7bc6c640 1 -- 172.21.15.60:0/3146685148 shutdown_connections 2023-08-24T10:08:16.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.841+0000 7f9c7bc6c640 1 --2- 172.21.15.60:0/3146685148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c74132630 0x7f9c74134a50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:16.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.841+0000 7f9c7bc6c640 1 -- 172.21.15.60:0/3146685148 >> 172.21.15.60:0/3146685148 conn(0x7f9c7409e110 msgr2=0x7f9c7409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:16.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.841+0000 7f9c7bc6c640 1 -- 172.21.15.60:0/3146685148 shutdown_connections 2023-08-24T10:08:16.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.841+0000 7f9c7bc6c640 1 -- 172.21.15.60:0/3146685148 wait complete. 2023-08-24T10:08:16.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.841+0000 7f9c7bc6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:16.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.842+0000 7f9c7bc6c640 1 Processor -- start 2023-08-24T10:08:16.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.842+0000 7f9c7bc6c640 1 -- start start 2023-08-24T10:08:16.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.843+0000 7f9c7bc6c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c74096860 0x7f9c741281a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:16.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.843+0000 7f9c7bc6c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7412bd50 con 0x7f9c74096860 2023-08-24T10:08:16.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.843+0000 7f9c79a08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c74096860 0x7f9c741281a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:16.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.843+0000 7f9c79a08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c74096860 0x7f9c741281a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48008/0 (socket says 172.21.15.60:48008) 2023-08-24T10:08:16.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.843+0000 7f9c79a08640 1 -- 172.21.15.60:0/1169976136 learned_addr learned my addr 172.21.15.60:0/1169976136 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:16.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.843+0000 7f9c79a08640 1 -- 172.21.15.60:0/1169976136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c74127b60 con 0x7f9c74096860 2023-08-24T10:08:16.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.844+0000 7f9c79a08640 1 --2- 172.21.15.60:0/1169976136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c74096860 0x7f9c741281a0 crc :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:16.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.844+0000 7f9c6a7fc640 1 -- 172.21.15.60:0/1169976136 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9c7000c7e0 con 0x7f9c74096860 2023-08-24T10:08:16.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.844+0000 7f9c6a7fc640 1 -- 172.21.15.60:0/1169976136 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f9c7000c940 con 0x7f9c74096860 2023-08-24T10:08:16.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.844+0000 7f9c7bc6c640 1 -- 172.21.15.60:0/1169976136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c74127e00 con 0x7f9c74096860 2023-08-24T10:08:16.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.844+0000 7f9c6a7fc640 1 -- 172.21.15.60:0/1169976136 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9c7000cbe0 con 0x7f9c74096860 2023-08-24T10:08:16.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.844+0000 7f9c7bc6c640 1 -- 172.21.15.60:0/1169976136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7412b0f0 con 0x7f9c74096860 2023-08-24T10:08:16.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.845+0000 7f9c6a7fc640 1 -- 172.21.15.60:0/1169976136 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f9c70011310 con 0x7f9c74096860 2023-08-24T10:08:16.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.845+0000 7f9c7bc6c640 1 -- 172.21.15.60:0/1169976136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c7412a460 con 0x7f9c74096860 2023-08-24T10:08:16.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.845+0000 7f9c6a7fc640 1 --2- 172.21.15.60:0/1169976136 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9c58037130 0x7f9c58039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:16.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.845+0000 7f9c6a7fc640 1 -- 172.21.15.60:0/1169976136 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f9c70046b40 con 0x7f9c74096860 2023-08-24T10:08:16.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.845+0000 7f9c79207640 1 --2- 172.21.15.60:0/1169976136 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9c58037130 0x7f9c58039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:16.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.846+0000 7f9c79207640 1 --2- 172.21.15.60:0/1169976136 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9c58037130 0x7f9c58039640 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:16.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:16.850+0000 7f9c6a7fc640 1 -- 172.21.15.60:0/1169976136 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f9c7000ad00 con 0x7f9c74096860 2023-08-24T10:08:17.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.024+0000 7f9c7bc6c640 1 -- 172.21.15.60:0/1169976136 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9c7406d190 con 0x7f9c58037130 2023-08-24T10:08:17.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.025+0000 7f9c6a7fc640 1 -- 172.21.15.60:0/1169976136 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f9c64009df0 con 0x7f9c58037130 2023-08-24T10:08:17.028 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:17.029 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:17.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.027+0000 7f9c7bc6c640 1 -- 172.21.15.60:0/1169976136 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9c58037130 msgr2=0x7f9c58039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:17.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.027+0000 7f9c7bc6c640 1 --2- 172.21.15.60:0/1169976136 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9c58037130 0x7f9c58039640 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:17.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.027+0000 7f9c7bc6c640 1 -- 172.21.15.60:0/1169976136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c74096860 msgr2=0x7f9c741281a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:17.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.027+0000 7f9c7bc6c640 1 --2- 172.21.15.60:0/1169976136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c74096860 0x7f9c741281a0 crc :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:17.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.027+0000 7f9c7bc6c640 1 -- 172.21.15.60:0/1169976136 shutdown_connections 2023-08-24T10:08:17.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.027+0000 7f9c7bc6c640 1 --2- 172.21.15.60:0/1169976136 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9c58037130 0x7f9c58039640 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:17.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.027+0000 7f9c7bc6c640 1 --2- 172.21.15.60:0/1169976136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c74096860 0x7f9c741281a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:17.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.027+0000 7f9c7bc6c640 1 -- 172.21.15.60:0/1169976136 >> 172.21.15.60:0/1169976136 conn(0x7f9c7409e110 msgr2=0x7f9c740950c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:17.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.027+0000 7f9c7bc6c640 1 -- 172.21.15.60:0/1169976136 shutdown_connections 2023-08-24T10:08:17.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.027+0000 7f9c7bc6c640 1 -- 172.21.15.60:0/1169976136 wait complete. 2023-08-24T10:08:17.039 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2023-08-24T10:08:15.532776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:17.041 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:17.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.151+0000 7f6bd171e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:17.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.155+0000 7f6bd171e640 1 Processor -- start 2023-08-24T10:08:17.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.155+0000 7f6bd171e640 1 -- start start 2023-08-24T10:08:17.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.155+0000 7f6bd171e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6bcc06d300 0x7f6bcc12bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:17.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.155+0000 7f6bd171e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bcc099440 con 0x7f6bcc12c1c0 2023-08-24T10:08:17.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.155+0000 7f6bd171e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bcc09a050 con 0x7f6bcc06d300 2023-08-24T10:08:17.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.156+0000 7f6bcaffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6bcc06d300 0x7f6bcc12bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:17.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.156+0000 7f6bcaffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6bcc06d300 0x7f6bcc12bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48020/0 (socket says 172.21.15.60:48020) 2023-08-24T10:08:17.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.156+0000 7f6bc27fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6bcc12c1c0 0x7f6bcc12ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36786/0 (socket says 172.21.15.60:36786) 2023-08-24T10:08:17.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.156+0000 7f6bcaffd640 1 -- 172.21.15.60:0/4235233955 learned_addr learned my addr 172.21.15.60:0/4235233955 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:17.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.156+0000 7f6bcaffd640 1 -- 172.21.15.60:0/4235233955 >> v1:172.21.15.60:6789/0 conn(0x7f6bcc12c1c0 legacy=0x7f6bcc12ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:17.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.156+0000 7f6bcaffd640 1 -- 172.21.15.60:0/4235233955 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc12b9d0 con 0x7f6bcc06d300 2023-08-24T10:08:17.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.156+0000 7f6bcaffd640 1 --2- 172.21.15.60:0/4235233955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bcc06d300 0x7f6bcc12bc80 crc :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02994d5f331eddf server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:17.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.156+0000 7f6bca7fc640 1 -- 172.21.15.60:0/4235233955 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6bbc00ad60 con 0x7f6bcc06d300 2023-08-24T10:08:17.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.156+0000 7f6bca7fc640 1 -- 172.21.15.60:0/4235233955 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f6bbc00f340 con 0x7f6bcc06d300 2023-08-24T10:08:17.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.156+0000 7f6bca7fc640 1 -- 172.21.15.60:0/4235233955 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6bbc00f5e0 con 0x7f6bcc06d300 2023-08-24T10:08:17.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.157+0000 7f6bd171e640 1 -- 172.21.15.60:0/4235233955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bcc06d300 msgr2=0x7f6bcc12bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:17.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.157+0000 7f6bd171e640 1 --2- 172.21.15.60:0/4235233955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bcc06d300 0x7f6bcc12bc80 crc :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:17.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.157+0000 7f6bd171e640 1 -- 172.21.15.60:0/4235233955 shutdown_connections 2023-08-24T10:08:17.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.157+0000 7f6bd171e640 1 --2- 172.21.15.60:0/4235233955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bcc06d300 0x7f6bcc12bc80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:17.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.157+0000 7f6bd171e640 1 -- 172.21.15.60:0/4235233955 >> 172.21.15.60:0/4235233955 conn(0x7f6bcc09e110 msgr2=0x7f6bcc09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:17.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.157+0000 7f6bd171e640 1 -- 172.21.15.60:0/4235233955 shutdown_connections 2023-08-24T10:08:17.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.157+0000 7f6bd171e640 1 -- 172.21.15.60:0/4235233955 wait complete. 2023-08-24T10:08:17.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.157+0000 7f6bd171e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:17.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.158+0000 7f6bd171e640 1 Processor -- start 2023-08-24T10:08:17.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.158+0000 7f6bd171e640 1 -- start start 2023-08-24T10:08:17.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.159+0000 7f6bd171e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bcc06d300 0x7f6bcc070c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:17.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.159+0000 7f6bd171e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc08a080 con 0x7f6bcc06d300 2023-08-24T10:08:17.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.159+0000 7f6bcaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bcc06d300 0x7f6bcc070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:17.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.159+0000 7f6bcaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bcc06d300 0x7f6bcc070c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48022/0 (socket says 172.21.15.60:48022) 2023-08-24T10:08:17.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.159+0000 7f6bcaffd640 1 -- 172.21.15.60:0/202458929 learned_addr learned my addr 172.21.15.60:0/202458929 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:17.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.159+0000 7f6bcaffd640 1 -- 172.21.15.60:0/202458929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc089280 con 0x7f6bcc06d300 2023-08-24T10:08:17.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.160+0000 7f6bcaffd640 1 --2- 172.21.15.60:0/202458929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bcc06d300 0x7f6bcc070c00 crc :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:17.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.160+0000 7f6bc3fff640 1 -- 172.21.15.60:0/202458929 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6bbc00fd70 con 0x7f6bcc06d300 2023-08-24T10:08:17.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.160+0000 7f6bc3fff640 1 -- 172.21.15.60:0/202458929 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f6bbc00fed0 con 0x7f6bcc06d300 2023-08-24T10:08:17.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.160+0000 7f6bc3fff640 1 -- 172.21.15.60:0/202458929 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6bbc011550 con 0x7f6bcc06d300 2023-08-24T10:08:17.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.160+0000 7f6bd171e640 1 -- 172.21.15.60:0/202458929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bcc0708f0 con 0x7f6bcc06d300 2023-08-24T10:08:17.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.160+0000 7f6bd171e640 1 -- 172.21.15.60:0/202458929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bcc08b3b0 con 0x7f6bcc06d300 2023-08-24T10:08:17.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.161+0000 7f6bc3fff640 1 -- 172.21.15.60:0/202458929 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f6bbc011750 con 0x7f6bcc06d300 2023-08-24T10:08:17.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.161+0000 7f6bd171e640 1 -- 172.21.15.60:0/202458929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bcc06d9f0 con 0x7f6bcc06d300 2023-08-24T10:08:17.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.161+0000 7f6bc3fff640 1 --2- 172.21.15.60:0/202458929 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6bac0370e0 0x7f6bac0395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:17.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.161+0000 7f6bc3fff640 1 -- 172.21.15.60:0/202458929 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f6bbc017040 con 0x7f6bcc06d300 2023-08-24T10:08:17.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.161+0000 7f6bc27fc640 1 --2- 172.21.15.60:0/202458929 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6bac0370e0 0x7f6bac0395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:17.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.162+0000 7f6bc27fc640 1 --2- 172.21.15.60:0/202458929 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6bac0370e0 0x7f6bac0395f0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:17.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.165+0000 7f6bc3fff640 1 -- 172.21.15.60:0/202458929 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f6bbc0103b0 con 0x7f6bcc06d300 2023-08-24T10:08:17.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.337+0000 7f6bd171e640 1 -- 172.21.15.60:0/202458929 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6bcc08b910 con 0x7f6bac0370e0 2023-08-24T10:08:17.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.338+0000 7f6bc3fff640 1 -- 172.21.15.60:0/202458929 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f6bcc08b910 con 0x7f6bac0370e0 2023-08-24T10:08:17.340 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:17.340 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:17.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.340+0000 7f6bd171e640 1 -- 172.21.15.60:0/202458929 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6bac0370e0 msgr2=0x7f6bac0395f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:17.341 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.340+0000 7f6bd171e640 1 --2- 172.21.15.60:0/202458929 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6bac0370e0 0x7f6bac0395f0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.340+0000 7f6bd171e640 1 -- 172.21.15.60:0/202458929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bcc06d300 msgr2=0x7f6bcc070c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.340+0000 7f6bd171e640 1 --2- 172.21.15.60:0/202458929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bcc06d300 0x7f6bcc070c00 crc :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.340+0000 7f6bd171e640 1 -- 172.21.15.60:0/202458929 shutdown_connections 2023-08-24T10:08:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.340+0000 7f6bd171e640 1 --2- 172.21.15.60:0/202458929 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6bac0370e0 0x7f6bac0395f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.340+0000 7f6bd171e640 1 --2- 172.21.15.60:0/202458929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bcc06d300 0x7f6bcc070c00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.340+0000 7f6bd171e640 1 -- 172.21.15.60:0/202458929 >> 172.21.15.60:0/202458929 conn(0x7f6bcc09e110 msgr2=0x7f6bcc09bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:17.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.340+0000 7f6bd171e640 1 -- 172.21.15.60:0/202458929 shutdown_connections 2023-08-24T10:08:17.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.340+0000 7f6bd171e640 1 -- 172.21.15.60:0/202458929 wait complete. 2023-08-24T10:08:17.354 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2023-08-24T10:08:15.532776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:17.357 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:08:17.358 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:17.358 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:08:17.359 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:17.360 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:08:17.360 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:17.361 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:08:17.362 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:17.364 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:08:17.364 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:17.365 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:08:17.365 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:17.365 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:08:17.365 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:17.366 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:08:17.366 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:17.366 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:08:17.366 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:17.367 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:08:17.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.477+0000 7fcf0839e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:17.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.479+0000 7fcf0839e640 1 Processor -- start 2023-08-24T10:08:17.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.480+0000 7fcf0839e640 1 -- start start 2023-08-24T10:08:17.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.480+0000 7fcf0839e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcf0012f610 0x7fcf0012fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:17.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.480+0000 7fcf0839e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf0009d750 con 0x7fcf001326f0 2023-08-24T10:08:17.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.480+0000 7fcf0839e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf0009d980 con 0x7fcf0012f610 2023-08-24T10:08:17.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.481+0000 7fcf0613a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcf0012f610 0x7fcf0012fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:17.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.481+0000 7fcf0613a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcf0012f610 0x7fcf0012fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48030/0 (socket says 172.21.15.60:48030) 2023-08-24T10:08:17.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.481+0000 7fcf0613a640 1 -- 172.21.15.60:0/2632024519 learned_addr learned my addr 172.21.15.60:0/2632024519 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:17.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.481+0000 7fcf05939640 1 --1- 172.21.15.60:0/2632024519 >> v1:172.21.15.60:6789/0 conn(0x7fcf001326f0 0x7fcf00132b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36798/0 (socket says 172.21.15.60:36798) 2023-08-24T10:08:17.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.481+0000 7fcf0613a640 1 -- 172.21.15.60:0/2632024519 >> v1:172.21.15.60:6789/0 conn(0x7fcf001326f0 legacy=0x7fcf00132b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:17.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.481+0000 7fcf0613a640 1 -- 172.21.15.60:0/2632024519 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0009bde0 con 0x7fcf0012f610 2023-08-24T10:08:17.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.481+0000 7fcf0613a640 1 --2- 172.21.15.60:0/2632024519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf0012f610 0x7fcf0012fa20 crc :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3559e4b7ec06ff36 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:17.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.481+0000 7fcf05138640 1 -- 172.21.15.60:0/2632024519 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcef4009dc0 con 0x7fcf0012f610 2023-08-24T10:08:17.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.481+0000 7fcf05138640 1 -- 172.21.15.60:0/2632024519 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fcef400e230 con 0x7fcf0012f610 2023-08-24T10:08:17.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.481+0000 7fcf05138640 1 -- 172.21.15.60:0/2632024519 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcef400e4d0 con 0x7fcf0012f610 2023-08-24T10:08:17.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.482+0000 7fcf0839e640 1 -- 172.21.15.60:0/2632024519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf0012f610 msgr2=0x7fcf0012fa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:17.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.482+0000 7fcf0839e640 1 --2- 172.21.15.60:0/2632024519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf0012f610 0x7fcf0012fa20 crc :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:17.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.482+0000 7fcf0839e640 1 -- 172.21.15.60:0/2632024519 shutdown_connections 2023-08-24T10:08:17.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.482+0000 7fcf0839e640 1 --2- 172.21.15.60:0/2632024519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf0012f610 0x7fcf0012fa20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:17.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.482+0000 7fcf0839e640 1 -- 172.21.15.60:0/2632024519 >> 172.21.15.60:0/2632024519 conn(0x7fcf000a2500 msgr2=0x7fcf000a2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:17.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.482+0000 7fcf0839e640 1 -- 172.21.15.60:0/2632024519 shutdown_connections 2023-08-24T10:08:17.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.482+0000 7fcf0839e640 1 -- 172.21.15.60:0/2632024519 wait complete. 2023-08-24T10:08:17.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.482+0000 7fcf0839e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:17.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.483+0000 7fcf0839e640 1 Processor -- start 2023-08-24T10:08:17.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.483+0000 7fcf0839e640 1 -- start start 2023-08-24T10:08:17.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.484+0000 7fcf0839e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf0012f610 0x7fcf0009fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:17.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.484+0000 7fcf0839e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0012cc60 con 0x7fcf0012f610 2023-08-24T10:08:17.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.484+0000 7fcf0613a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf0012f610 0x7fcf0009fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:17.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.484+0000 7fcf0613a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf0012f610 0x7fcf0009fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48046/0 (socket says 172.21.15.60:48046) 2023-08-24T10:08:17.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.484+0000 7fcf0613a640 1 -- 172.21.15.60:0/3086806721 learned_addr learned my addr 172.21.15.60:0/3086806721 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:17.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.484+0000 7fcf0613a640 1 -- 172.21.15.60:0/3086806721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0006a1f0 con 0x7fcf0012f610 2023-08-24T10:08:17.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.484+0000 7fcf0613a640 1 --2- 172.21.15.60:0/3086806721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf0012f610 0x7fcf0009fca0 crc :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:17.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.484+0000 7fcef2ffd640 1 -- 172.21.15.60:0/3086806721 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcef4014540 con 0x7fcf0012f610 2023-08-24T10:08:17.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.484+0000 7fcef2ffd640 1 -- 172.21.15.60:0/3086806721 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fcef40146a0 con 0x7fcf0012f610 2023-08-24T10:08:17.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.484+0000 7fcf0839e640 1 -- 172.21.15.60:0/3086806721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0009f7e0 con 0x7fcf0012f610 2023-08-24T10:08:17.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.484+0000 7fcef2ffd640 1 -- 172.21.15.60:0/3086806721 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcef4014940 con 0x7fcf0012f610 2023-08-24T10:08:17.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.484+0000 7fcf0839e640 1 -- 172.21.15.60:0/3086806721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0009f4d0 con 0x7fcf0012f610 2023-08-24T10:08:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.485+0000 7fcef2ffd640 1 -- 172.21.15.60:0/3086806721 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fcef4014af0 con 0x7fcf0012f610 2023-08-24T10:08:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.485+0000 7fcf0839e640 1 -- 172.21.15.60:0/3086806721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf00061800 con 0x7fcf0012f610 2023-08-24T10:08:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.486+0000 7fcef2ffd640 1 --2- 172.21.15.60:0/3086806721 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcee4037130 0x7fcee4039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.486+0000 7fcef2ffd640 1 -- 172.21.15.60:0/3086806721 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fcef4019040 con 0x7fcf0012f610 2023-08-24T10:08:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.486+0000 7fcf05939640 1 --2- 172.21.15.60:0/3086806721 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcee4037130 0x7fcee4039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.486+0000 7fcf05939640 1 --2- 172.21.15.60:0/3086806721 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcee4037130 0x7fcee4039640 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:17.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.490+0000 7fcef2ffd640 1 -- 172.21.15.60:0/3086806721 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fcef400fb00 con 0x7fcf0012f610 2023-08-24T10:08:17.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.731+0000 7fcf0839e640 1 -- 172.21.15.60:0/3086806721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fcf0006a1f0 con 0x7fcf0012f610 2023-08-24T10:08:17.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.732+0000 7fcef2ffd640 1 -- 172.21.15.60:0/3086806721 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7fcf0006a1f0 con 0x7fcf0012f610 2023-08-24T10:08:17.733 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:17.733 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":208,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:08:17.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.733+0000 7fcf0839e640 1 -- 172.21.15.60:0/3086806721 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcee4037130 msgr2=0x7fcee4039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:17.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.733+0000 7fcf0839e640 1 --2- 172.21.15.60:0/3086806721 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcee4037130 0x7fcee4039640 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:17.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.733+0000 7fcf0839e640 1 -- 172.21.15.60:0/3086806721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf0012f610 msgr2=0x7fcf0009fca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:17.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.733+0000 7fcf0839e640 1 --2- 172.21.15.60:0/3086806721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf0012f610 0x7fcf0009fca0 crc :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:17.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.733+0000 7fcf0839e640 1 -- 172.21.15.60:0/3086806721 shutdown_connections 2023-08-24T10:08:17.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.733+0000 7fcf0839e640 1 --2- 172.21.15.60:0/3086806721 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcee4037130 0x7fcee4039640 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:17.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.733+0000 7fcf0839e640 1 --2- 172.21.15.60:0/3086806721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf0012f610 0x7fcf0009fca0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:17.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.733+0000 7fcf0839e640 1 -- 172.21.15.60:0/3086806721 >> 172.21.15.60:0/3086806721 conn(0x7fcf000a2500 msgr2=0x7fcf001ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:17.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.734+0000 7fcf0839e640 1 -- 172.21.15.60:0/3086806721 shutdown_connections 2023-08-24T10:08:17.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.734+0000 7fcf0839e640 1 -- 172.21.15.60:0/3086806721 wait complete. 2023-08-24T10:08:17.746 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:08:17.746 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:17.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.854+0000 7f154ed1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:17.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.858+0000 7f154ed1e640 1 Processor -- start 2023-08-24T10:08:17.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.858+0000 7f154ed1e640 1 -- start start 2023-08-24T10:08:17.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.859+0000 7f154ed1e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f154806d300 0x7f154812bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:17.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.859+0000 7f154ed1e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1548099440 con 0x7f154812c1c0 2023-08-24T10:08:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.859+0000 7f154ed1e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f154809a050 con 0x7f154806d300 2023-08-24T10:08:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.859+0000 7f154caba640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f154806d300 0x7f154812bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.859+0000 7f154caba640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f154806d300 0x7f154812bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48052/0 (socket says 172.21.15.60:48052) 2023-08-24T10:08:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.859+0000 7f154caba640 1 -- 172.21.15.60:0/1543829628 learned_addr learned my addr 172.21.15.60:0/1543829628 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.859+0000 7f154caba640 1 -- 172.21.15.60:0/1543829628 >> v1:172.21.15.60:6789/0 conn(0x7f154812c1c0 legacy=0x7f154812ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.859+0000 7f154caba640 1 -- 172.21.15.60:0/1543829628 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154812b9d0 con 0x7f154806d300 2023-08-24T10:08:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.859+0000 7f154caba640 1 --2- 172.21.15.60:0/1543829628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f154806d300 0x7f154812bc80 crc :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64a8b8936f72ef5 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.860+0000 7f15477fe640 1 -- 172.21.15.60:0/1543829628 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1540009dd0 con 0x7f154806d300 2023-08-24T10:08:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.860+0000 7f15477fe640 1 -- 172.21.15.60:0/1543829628 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f154000e230 con 0x7f154806d300 2023-08-24T10:08:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.860+0000 7f15477fe640 1 -- 172.21.15.60:0/1543829628 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f154000e4d0 con 0x7f154806d300 2023-08-24T10:08:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.860+0000 7f154ed1e640 1 -- 172.21.15.60:0/1543829628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f154806d300 msgr2=0x7f154812bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.860+0000 7f154ed1e640 1 --2- 172.21.15.60:0/1543829628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f154806d300 0x7f154812bc80 crc :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.860+0000 7f154ed1e640 1 -- 172.21.15.60:0/1543829628 shutdown_connections 2023-08-24T10:08:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.860+0000 7f154ed1e640 1 --2- 172.21.15.60:0/1543829628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f154806d300 0x7f154812bc80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.860+0000 7f154ed1e640 1 -- 172.21.15.60:0/1543829628 >> 172.21.15.60:0/1543829628 conn(0x7f154809e110 msgr2=0x7f154809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.860+0000 7f154ed1e640 1 -- 172.21.15.60:0/1543829628 shutdown_connections 2023-08-24T10:08:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.860+0000 7f154ed1e640 1 -- 172.21.15.60:0/1543829628 wait complete. 2023-08-24T10:08:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.861+0000 7f154ed1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.861+0000 7f154ed1e640 1 Processor -- start 2023-08-24T10:08:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.861+0000 7f154ed1e640 1 -- start start 2023-08-24T10:08:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.862+0000 7f154ed1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f154806d300 0x7f1548096610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.862+0000 7f154ed1e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1548130540 con 0x7f154806d300 2023-08-24T10:08:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.862+0000 7f154caba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f154806d300 0x7f1548096610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.862+0000 7f154caba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f154806d300 0x7f1548096610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48068/0 (socket says 172.21.15.60:48068) 2023-08-24T10:08:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.862+0000 7f154caba640 1 -- 172.21.15.60:0/456044632 learned_addr learned my addr 172.21.15.60:0/456044632 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.863+0000 7f154caba640 1 -- 172.21.15.60:0/456044632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1548070840 con 0x7f154806d300 2023-08-24T10:08:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.863+0000 7f154caba640 1 --2- 172.21.15.60:0/456044632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f154806d300 0x7f1548096610 crc :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.863+0000 7f15457fa640 1 -- 172.21.15.60:0/456044632 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1540014410 con 0x7f154806d300 2023-08-24T10:08:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.863+0000 7f15457fa640 1 -- 172.21.15.60:0/456044632 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1540014570 con 0x7f154806d300 2023-08-24T10:08:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.863+0000 7f154ed1e640 1 -- 172.21.15.60:0/456044632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1548135a00 con 0x7f154806d300 2023-08-24T10:08:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.863+0000 7f15457fa640 1 -- 172.21.15.60:0/456044632 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1540014810 con 0x7f154806d300 2023-08-24T10:08:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.863+0000 7f154ed1e640 1 -- 172.21.15.60:0/456044632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1548071860 con 0x7f154806d300 2023-08-24T10:08:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.864+0000 7f15457fa640 1 -- 172.21.15.60:0/456044632 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f15400149c0 con 0x7f154806d300 2023-08-24T10:08:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.864+0000 7f154ed1e640 1 -- 172.21.15.60:0/456044632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154806fef0 con 0x7f154806d300 2023-08-24T10:08:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.864+0000 7f15457fa640 1 --2- 172.21.15.60:0/456044632 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1528037130 0x7f1528039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.864+0000 7f15457fa640 1 -- 172.21.15.60:0/456044632 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f1540019040 con 0x7f154806d300 2023-08-24T10:08:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.864+0000 7f1547fff640 1 --2- 172.21.15.60:0/456044632 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1528037130 0x7f1528039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.865+0000 7f1547fff640 1 --2- 172.21.15.60:0/456044632 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1528037130 0x7f1528039640 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:17.869+0000 7f15457fa640 1 -- 172.21.15.60:0/456044632 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f154000f8f0 con 0x7f154806d300 2023-08-24T10:08:18.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:18.041+0000 7f154ed1e640 1 -- 172.21.15.60:0/456044632 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f15481303d0 con 0x7f1528037130 2023-08-24T10:08:18.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:18.043+0000 7f15457fa640 1 -- 172.21.15.60:0/456044632 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f15481303d0 con 0x7f1528037130 2023-08-24T10:08:18.044 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:18.044 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:18.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:18.044+0000 7f154ed1e640 1 -- 172.21.15.60:0/456044632 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1528037130 msgr2=0x7f1528039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:18.044+0000 7f154ed1e640 1 --2- 172.21.15.60:0/456044632 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1528037130 0x7f1528039640 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:18.044+0000 7f154ed1e640 1 -- 172.21.15.60:0/456044632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f154806d300 msgr2=0x7f1548096610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:18.044+0000 7f154ed1e640 1 --2- 172.21.15.60:0/456044632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f154806d300 0x7f1548096610 crc :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:18.045+0000 7f154ed1e640 1 -- 172.21.15.60:0/456044632 shutdown_connections 2023-08-24T10:08:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:18.045+0000 7f154ed1e640 1 --2- 172.21.15.60:0/456044632 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1528037130 0x7f1528039640 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:18.045+0000 7f154ed1e640 1 --2- 172.21.15.60:0/456044632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f154806d300 0x7f1548096610 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:18.045+0000 7f154ed1e640 1 -- 172.21.15.60:0/456044632 >> 172.21.15.60:0/456044632 conn(0x7f154809e110 msgr2=0x7f15480978f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:18.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:18.045+0000 7f154ed1e640 1 -- 172.21.15.60:0/456044632 shutdown_connections 2023-08-24T10:08:18.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:18.045+0000 7f154ed1e640 1 -- 172.21.15.60:0/456044632 wait complete. 2023-08-24T10:08:18.057 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2023-08-24T10:08:17.533304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:19.990 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~193s 2023-08-24T10:08:19.990 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~193s 2023-08-24T10:08:19.990 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~193s 2023-08-24T10:08:19.990 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~193s 2023-08-24T10:08:21.060 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:21.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.180+0000 7fa3cd39e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:21.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.184+0000 7fa3cd39e640 1 Processor -- start 2023-08-24T10:08:21.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.184+0000 7fa3cd39e640 1 -- start start 2023-08-24T10:08:21.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.185+0000 7fa3cd39e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa3c8132630 0x7fa3c8134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:21.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.185+0000 7fa3cd39e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c806e320 con 0x7fa3c8096860 2023-08-24T10:08:21.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.185+0000 7fa3cd39e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c806db70 con 0x7fa3c8132630 2023-08-24T10:08:21.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.185+0000 7fa3c67fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa3c8132630 0x7fa3c8134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:21.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.185+0000 7fa3c67fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa3c8132630 0x7fa3c8134a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48084/0 (socket says 172.21.15.60:48084) 2023-08-24T10:08:21.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.185+0000 7fa3c67fc640 1 -- 172.21.15.60:0/1893935537 learned_addr learned my addr 172.21.15.60:0/1893935537 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:21.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.185+0000 7fa3c67fc640 1 -- 172.21.15.60:0/1893935537 >> v1:172.21.15.60:6789/0 conn(0x7fa3c8096860 legacy=0x7fa3c8096c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:21.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.185+0000 7fa3c67fc640 1 -- 172.21.15.60:0/1893935537 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c806cef0 con 0x7fa3c8132630 2023-08-24T10:08:21.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.185+0000 7fa3c67fc640 1 --2- 172.21.15.60:0/1893935537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c8132630 0x7fa3c8134a50 crc :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e342ee3ff147181a server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:21.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.186+0000 7fa3c5ffb640 1 -- 172.21.15.60:0/1893935537 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa3bc009dd0 con 0x7fa3c8132630 2023-08-24T10:08:21.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.186+0000 7fa3c5ffb640 1 -- 172.21.15.60:0/1893935537 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa3bc00e230 con 0x7fa3c8132630 2023-08-24T10:08:21.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.186+0000 7fa3c5ffb640 1 -- 172.21.15.60:0/1893935537 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa3bc00e4d0 con 0x7fa3c8132630 2023-08-24T10:08:21.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.186+0000 7fa3cd39e640 1 -- 172.21.15.60:0/1893935537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c8132630 msgr2=0x7fa3c8134a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:21.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.186+0000 7fa3cd39e640 1 --2- 172.21.15.60:0/1893935537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c8132630 0x7fa3c8134a50 crc :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:21.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.186+0000 7fa3cd39e640 1 -- 172.21.15.60:0/1893935537 shutdown_connections 2023-08-24T10:08:21.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.186+0000 7fa3cd39e640 1 --2- 172.21.15.60:0/1893935537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c8132630 0x7fa3c8134a50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:21.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.186+0000 7fa3cd39e640 1 -- 172.21.15.60:0/1893935537 >> 172.21.15.60:0/1893935537 conn(0x7fa3c809e110 msgr2=0x7fa3c809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:21.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.186+0000 7fa3cd39e640 1 -- 172.21.15.60:0/1893935537 shutdown_connections 2023-08-24T10:08:21.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.186+0000 7fa3cd39e640 1 -- 172.21.15.60:0/1893935537 wait complete. 2023-08-24T10:08:21.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.187+0000 7fa3cd39e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:21.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.188+0000 7fa3cd39e640 1 Processor -- start 2023-08-24T10:08:21.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.188+0000 7fa3cd39e640 1 -- start start 2023-08-24T10:08:21.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.188+0000 7fa3cd39e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c8096860 0x7fa3c808b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:21.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.188+0000 7fa3cd39e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c808c5e0 con 0x7fa3c8096860 2023-08-24T10:08:21.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.188+0000 7fa3c6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c8096860 0x7fa3c808b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:21.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.188+0000 7fa3c6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c8096860 0x7fa3c808b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48088/0 (socket says 172.21.15.60:48088) 2023-08-24T10:08:21.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.188+0000 7fa3c6ffd640 1 -- 172.21.15.60:0/2829156036 learned_addr learned my addr 172.21.15.60:0/2829156036 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:21.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.189+0000 7fa3c6ffd640 1 -- 172.21.15.60:0/2829156036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c808ab30 con 0x7fa3c8096860 2023-08-24T10:08:21.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.189+0000 7fa3c6ffd640 1 --2- 172.21.15.60:0/2829156036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c8096860 0x7fa3c808b080 crc :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:21.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.189+0000 7fa3af7fe640 1 -- 172.21.15.60:0/2829156036 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa3b800c7e0 con 0x7fa3c8096860 2023-08-24T10:08:21.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.189+0000 7fa3af7fe640 1 -- 172.21.15.60:0/2829156036 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa3b800c940 con 0x7fa3c8096860 2023-08-24T10:08:21.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.189+0000 7fa3cd39e640 1 -- 172.21.15.60:0/2829156036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c808ce00 con 0x7fa3c8096860 2023-08-24T10:08:21.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.189+0000 7fa3af7fe640 1 -- 172.21.15.60:0/2829156036 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa3b800cbe0 con 0x7fa3c8096860 2023-08-24T10:08:21.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.189+0000 7fa3cd39e640 1 -- 172.21.15.60:0/2829156036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c808e6f0 con 0x7fa3c8096860 2023-08-24T10:08:21.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.190+0000 7fa3af7fe640 1 -- 172.21.15.60:0/2829156036 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fa3b8011310 con 0x7fa3c8096860 2023-08-24T10:08:21.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.190+0000 7fa3cd39e640 1 -- 172.21.15.60:0/2829156036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c808df90 con 0x7fa3c8096860 2023-08-24T10:08:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.190+0000 7fa3af7fe640 1 --2- 172.21.15.60:0/2829156036 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa3b4037130 0x7fa3b4039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.190+0000 7fa3af7fe640 1 -- 172.21.15.60:0/2829156036 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fa3b8046a70 con 0x7fa3c8096860 2023-08-24T10:08:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.191+0000 7fa3c67fc640 1 --2- 172.21.15.60:0/2829156036 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa3b4037130 0x7fa3b4039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.191+0000 7fa3c67fc640 1 --2- 172.21.15.60:0/2829156036 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa3b4037130 0x7fa3b4039640 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.195+0000 7fa3af7fe640 1 -- 172.21.15.60:0/2829156036 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fa3b800ad00 con 0x7fa3c8096860 2023-08-24T10:08:21.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.366+0000 7fa3cd39e640 1 -- 172.21.15.60:0/2829156036 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa3c808a9c0 con 0x7fa3b4037130 2023-08-24T10:08:21.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.367+0000 7fa3af7fe640 1 -- 172.21.15.60:0/2829156036 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7fa3c808a9c0 con 0x7fa3b4037130 2023-08-24T10:08:21.369 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.369+0000 7fa3cd39e640 1 -- 172.21.15.60:0/2829156036 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa3b4037130 msgr2=0x7fa3b4039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.369+0000 7fa3cd39e640 1 --2- 172.21.15.60:0/2829156036 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa3b4037130 0x7fa3b4039640 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.369+0000 7fa3cd39e640 1 -- 172.21.15.60:0/2829156036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c8096860 msgr2=0x7fa3c808b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.369+0000 7fa3cd39e640 1 --2- 172.21.15.60:0/2829156036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c8096860 0x7fa3c808b080 crc :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.369+0000 7fa3cd39e640 1 -- 172.21.15.60:0/2829156036 shutdown_connections 2023-08-24T10:08:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.369+0000 7fa3cd39e640 1 --2- 172.21.15.60:0/2829156036 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa3b4037130 0x7fa3b4039640 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.369+0000 7fa3cd39e640 1 --2- 172.21.15.60:0/2829156036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c8096860 0x7fa3c808b080 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.369+0000 7fa3cd39e640 1 -- 172.21.15.60:0/2829156036 >> 172.21.15.60:0/2829156036 conn(0x7fa3c809e110 msgr2=0x7fa3c8097fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:21.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.369+0000 7fa3cd39e640 1 -- 172.21.15.60:0/2829156036 shutdown_connections 2023-08-24T10:08:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.369+0000 7fa3cd39e640 1 -- 172.21.15.60:0/2829156036 wait complete. 2023-08-24T10:08:21.381 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2023-08-24T10:08:19.533753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:21.382 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:21.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.490+0000 7f5ec455e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:21.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.494+0000 7f5ec455e640 1 Processor -- start 2023-08-24T10:08:21.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.494+0000 7f5ec455e640 1 -- start start 2023-08-24T10:08:21.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.495+0000 7f5ec455e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ebc06d300 0x7f5ebc12bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:21.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.495+0000 7f5ec455e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ebc099440 con 0x7f5ebc12c1c0 2023-08-24T10:08:21.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.495+0000 7f5ec455e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ebc09a050 con 0x7f5ebc06d300 2023-08-24T10:08:21.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.495+0000 7f5ec22fa640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ebc06d300 0x7f5ebc12bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:21.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.495+0000 7f5ec22fa640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ebc06d300 0x7f5ebc12bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48102/0 (socket says 172.21.15.60:48102) 2023-08-24T10:08:21.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.495+0000 7f5ec22fa640 1 -- 172.21.15.60:0/1722637612 learned_addr learned my addr 172.21.15.60:0/1722637612 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:21.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.495+0000 7f5ec22fa640 1 -- 172.21.15.60:0/1722637612 >> v1:172.21.15.60:6789/0 conn(0x7f5ebc12c1c0 legacy=0x7f5ebc12ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:21.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.495+0000 7f5ec22fa640 1 -- 172.21.15.60:0/1722637612 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ebc12b9d0 con 0x7f5ebc06d300 2023-08-24T10:08:21.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.495+0000 7f5ec22fa640 1 --2- 172.21.15.60:0/1722637612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ebc06d300 0x7f5ebc12bc80 crc :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c36a1dffca1bf86 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:21.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.496+0000 7f5ec12f8640 1 -- 172.21.15.60:0/1722637612 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5eb4009dd0 con 0x7f5ebc06d300 2023-08-24T10:08:21.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.496+0000 7f5ec12f8640 1 -- 172.21.15.60:0/1722637612 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5eb400e230 con 0x7f5ebc06d300 2023-08-24T10:08:21.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.496+0000 7f5ec12f8640 1 -- 172.21.15.60:0/1722637612 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5eb400e4d0 con 0x7f5ebc06d300 2023-08-24T10:08:21.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.496+0000 7f5ec455e640 1 -- 172.21.15.60:0/1722637612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ebc06d300 msgr2=0x7f5ebc12bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:21.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.496+0000 7f5ec455e640 1 --2- 172.21.15.60:0/1722637612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ebc06d300 0x7f5ebc12bc80 crc :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:21.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.496+0000 7f5ec455e640 1 -- 172.21.15.60:0/1722637612 shutdown_connections 2023-08-24T10:08:21.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.496+0000 7f5ec455e640 1 --2- 172.21.15.60:0/1722637612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ebc06d300 0x7f5ebc12bc80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:21.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.496+0000 7f5ec455e640 1 -- 172.21.15.60:0/1722637612 >> 172.21.15.60:0/1722637612 conn(0x7f5ebc09e110 msgr2=0x7f5ebc09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:21.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.496+0000 7f5ec455e640 1 -- 172.21.15.60:0/1722637612 shutdown_connections 2023-08-24T10:08:21.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.496+0000 7f5ec455e640 1 -- 172.21.15.60:0/1722637612 wait complete. 2023-08-24T10:08:21.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.497+0000 7f5ec455e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:21.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.498+0000 7f5ec455e640 1 Processor -- start 2023-08-24T10:08:21.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.498+0000 7f5ec455e640 1 -- start start 2023-08-24T10:08:21.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.498+0000 7f5ec455e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ebc06d300 0x7f5ebc096610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:21.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.498+0000 7f5ec455e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ebc130540 con 0x7f5ebc06d300 2023-08-24T10:08:21.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.498+0000 7f5ec22fa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ebc06d300 0x7f5ebc096610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:21.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.498+0000 7f5ec22fa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ebc06d300 0x7f5ebc096610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48116/0 (socket says 172.21.15.60:48116) 2023-08-24T10:08:21.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.498+0000 7f5ec22fa640 1 -- 172.21.15.60:0/1232068644 learned_addr learned my addr 172.21.15.60:0/1232068644 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:21.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.499+0000 7f5ec22fa640 1 -- 172.21.15.60:0/1232068644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ebc070840 con 0x7f5ebc06d300 2023-08-24T10:08:21.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.499+0000 7f5ec22fa640 1 --2- 172.21.15.60:0/1232068644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ebc06d300 0x7f5ebc096610 crc :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:21.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.499+0000 7f5ebaffd640 1 -- 172.21.15.60:0/1232068644 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5eb4018040 con 0x7f5ebc06d300 2023-08-24T10:08:21.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.499+0000 7f5ebaffd640 1 -- 172.21.15.60:0/1232068644 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5eb4012520 con 0x7f5ebc06d300 2023-08-24T10:08:21.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.499+0000 7f5ec455e640 1 -- 172.21.15.60:0/1232068644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ebc135a00 con 0x7f5ebc06d300 2023-08-24T10:08:21.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.499+0000 7f5ebaffd640 1 -- 172.21.15.60:0/1232068644 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5eb40127c0 con 0x7f5ebc06d300 2023-08-24T10:08:21.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.499+0000 7f5ec455e640 1 -- 172.21.15.60:0/1232068644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ebc071860 con 0x7f5ebc06d300 2023-08-24T10:08:21.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.500+0000 7f5ebaffd640 1 -- 172.21.15.60:0/1232068644 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f5eb4012970 con 0x7f5ebc06d300 2023-08-24T10:08:21.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.500+0000 7f5ec455e640 1 -- 172.21.15.60:0/1232068644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ebc06fef0 con 0x7f5ebc06d300 2023-08-24T10:08:21.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.500+0000 7f5ebaffd640 1 --2- 172.21.15.60:0/1232068644 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5ea8037130 0x7f5ea8039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:21.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.500+0000 7f5ebaffd640 1 -- 172.21.15.60:0/1232068644 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f5eb40493d0 con 0x7f5ebc06d300 2023-08-24T10:08:21.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.500+0000 7f5ec1af9640 1 --2- 172.21.15.60:0/1232068644 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5ea8037130 0x7f5ea8039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:21.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.501+0000 7f5ec1af9640 1 --2- 172.21.15.60:0/1232068644 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5ea8037130 0x7f5ea8039640 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:21.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.505+0000 7f5ebaffd640 1 -- 172.21.15.60:0/1232068644 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f5eb4017130 con 0x7f5ebc06d300 2023-08-24T10:08:21.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.677+0000 7f5ec455e640 1 -- 172.21.15.60:0/1232068644 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ebc1303d0 con 0x7f5ea8037130 2023-08-24T10:08:21.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.678+0000 7f5ebaffd640 1 -- 172.21.15.60:0/1232068644 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f5ebc1303d0 con 0x7f5ea8037130 2023-08-24T10:08:21.680 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:21.680 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.680+0000 7f5ec455e640 1 -- 172.21.15.60:0/1232068644 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5ea8037130 msgr2=0x7f5ea8039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.680+0000 7f5ec455e640 1 --2- 172.21.15.60:0/1232068644 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5ea8037130 0x7f5ea8039640 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.680+0000 7f5ec455e640 1 -- 172.21.15.60:0/1232068644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ebc06d300 msgr2=0x7f5ebc096610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.680+0000 7f5ec455e640 1 --2- 172.21.15.60:0/1232068644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ebc06d300 0x7f5ebc096610 crc :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.680+0000 7f5ec455e640 1 -- 172.21.15.60:0/1232068644 shutdown_connections 2023-08-24T10:08:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.680+0000 7f5ec455e640 1 --2- 172.21.15.60:0/1232068644 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5ea8037130 0x7f5ea8039640 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.680+0000 7f5ec455e640 1 --2- 172.21.15.60:0/1232068644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ebc06d300 0x7f5ebc096610 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.680+0000 7f5ec455e640 1 -- 172.21.15.60:0/1232068644 >> 172.21.15.60:0/1232068644 conn(0x7f5ebc09e110 msgr2=0x7f5ebc0978f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.680+0000 7f5ec455e640 1 -- 172.21.15.60:0/1232068644 shutdown_connections 2023-08-24T10:08:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.680+0000 7f5ec455e640 1 -- 172.21.15.60:0/1232068644 wait complete. 2023-08-24T10:08:21.692 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2023-08-24T10:08:21.534086+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:21.693 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:08:21.693 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:21.694 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:08:21.694 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:21.694 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:08:21.694 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:21.695 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:08:21.695 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:21.696 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:08:21.696 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:21.696 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:08:21.696 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:21.697 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:08:21.697 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:21.697 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:08:21.697 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:21.698 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:08:21.698 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:21.699 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:08:21.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.805+0000 7febf595e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:21.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.809+0000 7febf595e640 1 Processor -- start 2023-08-24T10:08:21.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.809+0000 7febf595e640 1 -- start start 2023-08-24T10:08:21.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.809+0000 7febf595e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7febf0135a60 0x7febf0137e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:21.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.809+0000 7febf595e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf009d750 con 0x7febf012f610 2023-08-24T10:08:21.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.809+0000 7febf595e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf0130130 con 0x7febf0135a60 2023-08-24T10:08:21.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.809+0000 7febeeffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7febf012f610 0x7febf012fa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36830/0 (socket says 172.21.15.60:36830) 2023-08-24T10:08:21.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.809+0000 7febeeffd640 1 -- 172.21.15.60:0/790411529 learned_addr learned my addr 172.21.15.60:0/790411529 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:08:21.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.809+0000 7febe7fff640 1 --2- 172.21.15.60:0/790411529 >> v2:172.21.15.60:3300/0 conn(0x7febf0135a60 0x7febf0137e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:21.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.810+0000 7febe7fff640 1 -- 172.21.15.60:0/790411529 >> v1:172.21.15.60:6789/0 conn(0x7febf012f610 legacy=0x7febf012fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:21.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.810+0000 7febe7fff640 1 -- 172.21.15.60:0/790411529 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf009d980 con 0x7febf0135a60 2023-08-24T10:08:21.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.810+0000 7febe7fff640 1 --2- 172.21.15.60:0/790411529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf0135a60 0x7febf0137e80 crc :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702f3c8b99a49e24 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:21.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.810+0000 7febee7fc640 1 -- 172.21.15.60:0/790411529 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7febd8009cd0 con 0x7febf0135a60 2023-08-24T10:08:21.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.810+0000 7febee7fc640 1 -- 172.21.15.60:0/790411529 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7febd8009e30 con 0x7febf0135a60 2023-08-24T10:08:21.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.810+0000 7febee7fc640 1 -- 172.21.15.60:0/790411529 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7febd800e360 con 0x7febf0135a60 2023-08-24T10:08:21.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.811+0000 7febf595e640 1 -- 172.21.15.60:0/790411529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf0135a60 msgr2=0x7febf0137e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:21.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.811+0000 7febf595e640 1 --2- 172.21.15.60:0/790411529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf0135a60 0x7febf0137e80 crc :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:21.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.811+0000 7febf595e640 1 -- 172.21.15.60:0/790411529 shutdown_connections 2023-08-24T10:08:21.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.811+0000 7febf595e640 1 --2- 172.21.15.60:0/790411529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf0135a60 0x7febf0137e80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:21.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.811+0000 7febf595e640 1 -- 172.21.15.60:0/790411529 >> 172.21.15.60:0/790411529 conn(0x7febf00a2500 msgr2=0x7febf00a2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:21.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.811+0000 7febf595e640 1 -- 172.21.15.60:0/790411529 shutdown_connections 2023-08-24T10:08:21.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.811+0000 7febf595e640 1 -- 172.21.15.60:0/790411529 wait complete. 2023-08-24T10:08:21.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.811+0000 7febf595e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:21.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.812+0000 7febf595e640 1 Processor -- start 2023-08-24T10:08:21.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.812+0000 7febf595e640 1 -- start start 2023-08-24T10:08:21.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.813+0000 7febf595e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf012f610 0x7febf00a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:21.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.813+0000 7febf595e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf00a08e0 con 0x7febf012f610 2023-08-24T10:08:21.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.813+0000 7febeeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf012f610 0x7febf00a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:21.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.813+0000 7febeeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf012f610 0x7febf00a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48146/0 (socket says 172.21.15.60:48146) 2023-08-24T10:08:21.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.813+0000 7febeeffd640 1 -- 172.21.15.60:0/1032210847 learned_addr learned my addr 172.21.15.60:0/1032210847 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:21.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.813+0000 7febeeffd640 1 -- 172.21.15.60:0/1032210847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf011d420 con 0x7febf012f610 2023-08-24T10:08:21.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.813+0000 7febeeffd640 1 --2- 172.21.15.60:0/1032210847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf012f610 0x7febf00a0d10 crc :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:21.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.813+0000 7febecff9640 1 -- 172.21.15.60:0/1032210847 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7febe00059d0 con 0x7febf012f610 2023-08-24T10:08:21.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.813+0000 7febecff9640 1 -- 172.21.15.60:0/1032210847 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7febe0005b30 con 0x7febf012f610 2023-08-24T10:08:21.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.813+0000 7febf595e640 1 -- 172.21.15.60:0/1032210847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf00a0540 con 0x7febf012f610 2023-08-24T10:08:21.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.813+0000 7febecff9640 1 -- 172.21.15.60:0/1032210847 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7febe0005dd0 con 0x7febf012f610 2023-08-24T10:08:21.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.813+0000 7febf595e640 1 -- 172.21.15.60:0/1032210847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf011c810 con 0x7febf012f610 2023-08-24T10:08:21.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.814+0000 7febecff9640 1 -- 172.21.15.60:0/1032210847 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7febe00070b0 con 0x7febf012f610 2023-08-24T10:08:21.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.814+0000 7febf595e640 1 -- 172.21.15.60:0/1032210847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf00a0730 con 0x7febf012f610 2023-08-24T10:08:21.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.814+0000 7febecff9640 1 --2- 172.21.15.60:0/1032210847 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7febd0037130 0x7febd0039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:21.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.814+0000 7febecff9640 1 -- 172.21.15.60:0/1032210847 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7febe0046470 con 0x7febf012f610 2023-08-24T10:08:21.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.815+0000 7febe7fff640 1 --2- 172.21.15.60:0/1032210847 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7febd0037130 0x7febd0039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:21.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.815+0000 7febe7fff640 1 --2- 172.21.15.60:0/1032210847 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7febd0037130 0x7febd0039640 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:21.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:21.819+0000 7febecff9640 1 -- 172.21.15.60:0/1032210847 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7febe000d330 con 0x7febf012f610 2023-08-24T10:08:22.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.060+0000 7febf595e640 1 -- 172.21.15.60:0/1032210847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7febf0061800 con 0x7febf012f610 2023-08-24T10:08:22.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.061+0000 7febecff9640 1 -- 172.21.15.60:0/1032210847 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7febe000d150 con 0x7febf012f610 2023-08-24T10:08:22.062 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:22.063 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":212,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:08:22.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.063+0000 7febf595e640 1 -- 172.21.15.60:0/1032210847 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7febd0037130 msgr2=0x7febd0039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:22.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.063+0000 7febf595e640 1 --2- 172.21.15.60:0/1032210847 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7febd0037130 0x7febd0039640 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:22.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.063+0000 7febf595e640 1 -- 172.21.15.60:0/1032210847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf012f610 msgr2=0x7febf00a0d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:22.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.063+0000 7febf595e640 1 --2- 172.21.15.60:0/1032210847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf012f610 0x7febf00a0d10 crc :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:22.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.063+0000 7febf595e640 1 -- 172.21.15.60:0/1032210847 shutdown_connections 2023-08-24T10:08:22.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.063+0000 7febf595e640 1 --2- 172.21.15.60:0/1032210847 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7febd0037130 0x7febd0039640 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:22.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.063+0000 7febf595e640 1 --2- 172.21.15.60:0/1032210847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7febf012f610 0x7febf00a0d10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:22.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.063+0000 7febf595e640 1 -- 172.21.15.60:0/1032210847 >> 172.21.15.60:0/1032210847 conn(0x7febf00a2500 msgr2=0x7febf0099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:22.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.063+0000 7febf595e640 1 -- 172.21.15.60:0/1032210847 shutdown_connections 2023-08-24T10:08:22.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.063+0000 7febf595e640 1 -- 172.21.15.60:0/1032210847 wait complete. 2023-08-24T10:08:22.074 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:08:22.075 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:22.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.184+0000 7fefdd263640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:22.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.188+0000 7fefdd263640 1 Processor -- start 2023-08-24T10:08:22.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.188+0000 7fefdd263640 1 -- start start 2023-08-24T10:08:22.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.189+0000 7fefdd263640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fefd812d870 0x7fefd8135740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:22.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.189+0000 7fefdd263640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd806e170 con 0x7fefd80a1c70 2023-08-24T10:08:22.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.189+0000 7fefdd263640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd80a2080 con 0x7fefd812d870 2023-08-24T10:08:22.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.189+0000 7fefd67fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fefd812d870 0x7fefd8135740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:22.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.189+0000 7fefd67fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fefd812d870 0x7fefd8135740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48150/0 (socket says 172.21.15.60:48150) 2023-08-24T10:08:22.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.189+0000 7fefd67fc640 1 -- 172.21.15.60:0/1869886271 learned_addr learned my addr 172.21.15.60:0/1869886271 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:22.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.189+0000 7fefd6ffd640 1 --1- 172.21.15.60:0/1869886271 >> v1:172.21.15.60:6789/0 conn(0x7fefd80a1c70 0x7fefd812d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36846/0 (socket says 172.21.15.60:36846) 2023-08-24T10:08:22.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.189+0000 7fefd67fc640 1 -- 172.21.15.60:0/1869886271 >> v1:172.21.15.60:6789/0 conn(0x7fefd80a1c70 legacy=0x7fefd812d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:22.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.189+0000 7fefd67fc640 1 -- 172.21.15.60:0/1869886271 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd812cec0 con 0x7fefd812d870 2023-08-24T10:08:22.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.189+0000 7fefd5ffb640 1 -- 172.21.15.60:0/1869886271 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509740230 0 0) 0x7fefd806e170 con 0x7fefd80a1c70 2023-08-24T10:08:22.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.190+0000 7fefd67fc640 1 --2- 172.21.15.60:0/1869886271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd812d870 0x7fefd8135740 crc :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88e1cae71c45007 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:22.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.190+0000 7fefd5ffb640 1 -- 172.21.15.60:0/1869886271 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fefcc009dd0 con 0x7fefd812d870 2023-08-24T10:08:22.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.190+0000 7fefd5ffb640 1 -- 172.21.15.60:0/1869886271 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fefcc00e230 con 0x7fefd812d870 2023-08-24T10:08:22.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.190+0000 7fefd5ffb640 1 -- 172.21.15.60:0/1869886271 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fefcc00e4d0 con 0x7fefd812d870 2023-08-24T10:08:22.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.190+0000 7fefdd263640 1 -- 172.21.15.60:0/1869886271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd812d870 msgr2=0x7fefd8135740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:22.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.190+0000 7fefdd263640 1 --2- 172.21.15.60:0/1869886271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd812d870 0x7fefd8135740 crc :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:22.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.190+0000 7fefdd263640 1 -- 172.21.15.60:0/1869886271 shutdown_connections 2023-08-24T10:08:22.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.190+0000 7fefdd263640 1 --2- 172.21.15.60:0/1869886271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd812d870 0x7fefd8135740 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:22.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.190+0000 7fefdd263640 1 -- 172.21.15.60:0/1869886271 >> 172.21.15.60:0/1869886271 conn(0x7fefd809f290 msgr2=0x7fefd809f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:22.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.190+0000 7fefdd263640 1 -- 172.21.15.60:0/1869886271 shutdown_connections 2023-08-24T10:08:22.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.190+0000 7fefdd263640 1 -- 172.21.15.60:0/1869886271 wait complete. 2023-08-24T10:08:22.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.191+0000 7fefdd263640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:22.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.192+0000 7fefdd263640 1 Processor -- start 2023-08-24T10:08:22.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.192+0000 7fefdd263640 1 -- start start 2023-08-24T10:08:22.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.192+0000 7fefdd263640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd80a1c70 0x7fefd81199f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:22.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.192+0000 7fefdd263640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd8119530 con 0x7fefd80a1c70 2023-08-24T10:08:22.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.192+0000 7fefd6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd80a1c70 0x7fefd81199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:22.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.192+0000 7fefd6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd80a1c70 0x7fefd81199f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48164/0 (socket says 172.21.15.60:48164) 2023-08-24T10:08:22.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.192+0000 7fefd6ffd640 1 -- 172.21.15.60:0/4199115808 learned_addr learned my addr 172.21.15.60:0/4199115808 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:22.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.193+0000 7fefd6ffd640 1 -- 172.21.15.60:0/4199115808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd812a7e0 con 0x7fefd80a1c70 2023-08-24T10:08:22.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.193+0000 7fefd6ffd640 1 --2- 172.21.15.60:0/4199115808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd80a1c70 0x7fefd81199f0 crc :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:22.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.193+0000 7fefbf7fe640 1 -- 172.21.15.60:0/4199115808 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fefc8004af0 con 0x7fefd80a1c70 2023-08-24T10:08:22.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.193+0000 7fefbf7fe640 1 -- 172.21.15.60:0/4199115808 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fefc8004c50 con 0x7fefd80a1c70 2023-08-24T10:08:22.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.193+0000 7fefbf7fe640 1 -- 172.21.15.60:0/4199115808 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fefc8007120 con 0x7fefd80a1c70 2023-08-24T10:08:22.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.193+0000 7fefdd263640 1 -- 172.21.15.60:0/4199115808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd8119190 con 0x7fefd80a1c70 2023-08-24T10:08:22.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.193+0000 7fefdd263640 1 -- 172.21.15.60:0/4199115808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd811ccd0 con 0x7fefd80a1c70 2023-08-24T10:08:22.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.194+0000 7fefbf7fe640 1 -- 172.21.15.60:0/4199115808 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fefc80072d0 con 0x7fefd80a1c70 2023-08-24T10:08:22.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.194+0000 7fefdd263640 1 -- 172.21.15.60:0/4199115808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd811ac60 con 0x7fefd80a1c70 2023-08-24T10:08:22.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.194+0000 7fefbf7fe640 1 --2- 172.21.15.60:0/4199115808 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fefc4037130 0x7fefc4039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:22.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.194+0000 7fefbf7fe640 1 -- 172.21.15.60:0/4199115808 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fefc8046570 con 0x7fefd80a1c70 2023-08-24T10:08:22.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.195+0000 7fefd67fc640 1 --2- 172.21.15.60:0/4199115808 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fefc4037130 0x7fefc4039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:22.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.195+0000 7fefd67fc640 1 --2- 172.21.15.60:0/4199115808 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fefc4037130 0x7fefc4039640 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.199+0000 7fefbf7fe640 1 -- 172.21.15.60:0/4199115808 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fefc800a940 con 0x7fefd80a1c70 2023-08-24T10:08:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.371+0000 7fefdd263640 1 -- 172.21.15.60:0/4199115808 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fefd811d2f0 con 0x7fefc4037130 2023-08-24T10:08:22.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.372+0000 7fefbf7fe640 1 -- 172.21.15.60:0/4199115808 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7fefd811d2f0 con 0x7fefc4037130 2023-08-24T10:08:22.373 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:22.374 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.375+0000 7fefdd263640 1 -- 172.21.15.60:0/4199115808 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fefc4037130 msgr2=0x7fefc4039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.375+0000 7fefdd263640 1 --2- 172.21.15.60:0/4199115808 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fefc4037130 0x7fefc4039640 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.375+0000 7fefdd263640 1 -- 172.21.15.60:0/4199115808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd80a1c70 msgr2=0x7fefd81199f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.375+0000 7fefdd263640 1 --2- 172.21.15.60:0/4199115808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd80a1c70 0x7fefd81199f0 crc :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.375+0000 7fefdd263640 1 -- 172.21.15.60:0/4199115808 shutdown_connections 2023-08-24T10:08:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.375+0000 7fefdd263640 1 --2- 172.21.15.60:0/4199115808 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fefc4037130 0x7fefc4039640 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.375+0000 7fefdd263640 1 --2- 172.21.15.60:0/4199115808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefd80a1c70 0x7fefd81199f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.375+0000 7fefdd263640 1 -- 172.21.15.60:0/4199115808 >> 172.21.15.60:0/4199115808 conn(0x7fefd809f290 msgr2=0x7fefd8070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.375+0000 7fefdd263640 1 -- 172.21.15.60:0/4199115808 shutdown_connections 2023-08-24T10:08:22.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:22.375+0000 7fefdd263640 1 -- 172.21.15.60:0/4199115808 wait complete. 2023-08-24T10:08:22.387 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2023-08-24T10:08:21.534086+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:25.193 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~198s 2023-08-24T10:08:25.193 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~198s 2023-08-24T10:08:25.194 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~198s 2023-08-24T10:08:25.194 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~198s 2023-08-24T10:08:25.388 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:25.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.509+0000 7f94bfd1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:25.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.513+0000 7f94bfd1e640 1 Processor -- start 2023-08-24T10:08:25.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.513+0000 7f94bfd1e640 1 -- start start 2023-08-24T10:08:25.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.514+0000 7f94bfd1e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f94b81337b0 0x7f94b8135bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:25.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.514+0000 7f94bfd1e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b812cfa0 con 0x7f94b812d360 2023-08-24T10:08:25.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.514+0000 7f94bfd1e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b812de80 con 0x7f94b81337b0 2023-08-24T10:08:25.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.514+0000 7f94bdaba640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f94b812d360 0x7f94b812d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36852/0 (socket says 172.21.15.60:36852) 2023-08-24T10:08:25.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.514+0000 7f94bdaba640 1 -- 172.21.15.60:0/1206345636 learned_addr learned my addr 172.21.15.60:0/1206345636 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:08:25.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.514+0000 7f94bd2b9640 1 --2- 172.21.15.60:0/1206345636 >> v2:172.21.15.60:3300/0 conn(0x7f94b81337b0 0x7f94b8135bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:25.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.515+0000 7f94bd2b9640 1 -- 172.21.15.60:0/1206345636 >> v1:172.21.15.60:6789/0 conn(0x7f94b812d360 legacy=0x7f94b812d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:25.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.515+0000 7f94bd2b9640 1 -- 172.21.15.60:0/1206345636 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b8070d70 con 0x7f94b81337b0 2023-08-24T10:08:25.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.515+0000 7f94bd2b9640 1 --2- 172.21.15.60:0/1206345636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94b81337b0 0x7f94b8135bd0 crc :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f10abc1c2db1a3 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:25.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.515+0000 7f94bcab8640 1 -- 172.21.15.60:0/1206345636 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f94ac009ce0 con 0x7f94b81337b0 2023-08-24T10:08:25.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.515+0000 7f94bcab8640 1 -- 172.21.15.60:0/1206345636 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f94ac009e40 con 0x7f94b81337b0 2023-08-24T10:08:25.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.515+0000 7f94bcab8640 1 -- 172.21.15.60:0/1206345636 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f94ac00e360 con 0x7f94b81337b0 2023-08-24T10:08:25.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.516+0000 7f94bfd1e640 1 -- 172.21.15.60:0/1206345636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94b81337b0 msgr2=0x7f94b8135bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:25.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.516+0000 7f94bfd1e640 1 --2- 172.21.15.60:0/1206345636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94b81337b0 0x7f94b8135bd0 crc :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:25.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.516+0000 7f94bfd1e640 1 -- 172.21.15.60:0/1206345636 shutdown_connections 2023-08-24T10:08:25.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.516+0000 7f94bfd1e640 1 --2- 172.21.15.60:0/1206345636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94b81337b0 0x7f94b8135bd0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:25.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.516+0000 7f94bfd1e640 1 -- 172.21.15.60:0/1206345636 >> 172.21.15.60:0/1206345636 conn(0x7f94b809e610 msgr2=0x7f94b809ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:25.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.516+0000 7f94bfd1e640 1 -- 172.21.15.60:0/1206345636 shutdown_connections 2023-08-24T10:08:25.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.516+0000 7f94bfd1e640 1 -- 172.21.15.60:0/1206345636 wait complete. 2023-08-24T10:08:25.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.516+0000 7f94bfd1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:25.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.517+0000 7f94bfd1e640 1 Processor -- start 2023-08-24T10:08:25.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.517+0000 7f94bfd1e640 1 -- start start 2023-08-24T10:08:25.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.518+0000 7f94bfd1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94b812d360 0x7f94b8119e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:25.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.518+0000 7f94bfd1e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b811c170 con 0x7f94b812d360 2023-08-24T10:08:25.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.518+0000 7f94bdaba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94b812d360 0x7f94b8119e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:25.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.518+0000 7f94bdaba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94b812d360 0x7f94b8119e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48176/0 (socket says 172.21.15.60:48176) 2023-08-24T10:08:25.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.518+0000 7f94bdaba640 1 -- 172.21.15.60:0/885405487 learned_addr learned my addr 172.21.15.60:0/885405487 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:25.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.518+0000 7f94bdaba640 1 -- 172.21.15.60:0/885405487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b8119c10 con 0x7f94b812d360 2023-08-24T10:08:25.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.518+0000 7f94bdaba640 1 --2- 172.21.15.60:0/885405487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94b812d360 0x7f94b8119e30 crc :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:25.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.518+0000 7f94b67fc640 1 -- 172.21.15.60:0/885405487 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f94a80070b0 con 0x7f94b812d360 2023-08-24T10:08:25.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.518+0000 7f94b67fc640 1 -- 172.21.15.60:0/885405487 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f94a8007210 con 0x7f94b812d360 2023-08-24T10:08:25.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.518+0000 7f94b67fc640 1 -- 172.21.15.60:0/885405487 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f94a80074b0 con 0x7f94b812d360 2023-08-24T10:08:25.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.518+0000 7f94bfd1e640 1 -- 172.21.15.60:0/885405487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b811b0d0 con 0x7f94b812d360 2023-08-24T10:08:25.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.518+0000 7f94bfd1e640 1 -- 172.21.15.60:0/885405487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b811d5c0 con 0x7f94b812d360 2023-08-24T10:08:25.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.519+0000 7f94b67fc640 1 -- 172.21.15.60:0/885405487 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f94a80076a0 con 0x7f94b812d360 2023-08-24T10:08:25.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.519+0000 7f94bfd1e640 1 -- 172.21.15.60:0/885405487 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b81199d0 con 0x7f94b812d360 2023-08-24T10:08:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.519+0000 7f94b67fc640 1 --2- 172.21.15.60:0/885405487 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f94a4037130 0x7f94a4039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.519+0000 7f94b67fc640 1 -- 172.21.15.60:0/885405487 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f94a8046ea0 con 0x7f94b812d360 2023-08-24T10:08:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.520+0000 7f94bd2b9640 1 --2- 172.21.15.60:0/885405487 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f94a4037130 0x7f94a4039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.520+0000 7f94bd2b9640 1 --2- 172.21.15.60:0/885405487 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f94a4037130 0x7f94a4039640 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.524+0000 7f94b67fc640 1 -- 172.21.15.60:0/885405487 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f94a800de80 con 0x7f94b812d360 2023-08-24T10:08:25.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.696+0000 7f94bfd1e640 1 -- 172.21.15.60:0/885405487 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f94b8070630 con 0x7f94a4037130 2023-08-24T10:08:25.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.697+0000 7f94b67fc640 1 -- 172.21.15.60:0/885405487 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f94ac012490 con 0x7f94a4037130 2023-08-24T10:08:25.698 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:25.699 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:25.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.699+0000 7f94bfd1e640 1 -- 172.21.15.60:0/885405487 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f94a4037130 msgr2=0x7f94a4039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:25.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.699+0000 7f94bfd1e640 1 --2- 172.21.15.60:0/885405487 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f94a4037130 0x7f94a4039640 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:25.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.699+0000 7f94bfd1e640 1 -- 172.21.15.60:0/885405487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94b812d360 msgr2=0x7f94b8119e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:25.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.699+0000 7f94bfd1e640 1 --2- 172.21.15.60:0/885405487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94b812d360 0x7f94b8119e30 crc :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:25.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.699+0000 7f94bfd1e640 1 -- 172.21.15.60:0/885405487 shutdown_connections 2023-08-24T10:08:25.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.699+0000 7f94bfd1e640 1 --2- 172.21.15.60:0/885405487 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f94a4037130 0x7f94a4039640 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:25.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.699+0000 7f94bfd1e640 1 --2- 172.21.15.60:0/885405487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94b812d360 0x7f94b8119e30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:25.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.699+0000 7f94bfd1e640 1 -- 172.21.15.60:0/885405487 >> 172.21.15.60:0/885405487 conn(0x7f94b809e610 msgr2=0x7f94b806f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:25.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.699+0000 7f94bfd1e640 1 -- 172.21.15.60:0/885405487 shutdown_connections 2023-08-24T10:08:25.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.699+0000 7f94bfd1e640 1 -- 172.21.15.60:0/885405487 wait complete. 2023-08-24T10:08:25.711 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2023-08-24T10:08:25.534851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:25.712 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:25.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.819+0000 7fd98895e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:25.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.823+0000 7fd98895e640 1 Processor -- start 2023-08-24T10:08:25.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.823+0000 7fd98895e640 1 -- start start 2023-08-24T10:08:25.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.824+0000 7fd98895e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd984131d40 0x7fd984134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:25.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.824+0000 7fd98895e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd984099440 con 0x7fd98406d300 2023-08-24T10:08:25.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.824+0000 7fd98895e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd98409a050 con 0x7fd984131d40 2023-08-24T10:08:25.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.824+0000 7fd981d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd984131d40 0x7fd984134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:25.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.824+0000 7fd981d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd984131d40 0x7fd984134160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48192/0 (socket says 172.21.15.60:48192) 2023-08-24T10:08:25.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.824+0000 7fd98259c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd98406d300 0x7fd98412bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36868/0 (socket says 172.21.15.60:36868) 2023-08-24T10:08:25.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.824+0000 7fd981d9b640 1 -- 172.21.15.60:0/1848490470 learned_addr learned my addr 172.21.15.60:0/1848490470 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:25.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.824+0000 7fd981d9b640 1 -- 172.21.15.60:0/1848490470 >> v1:172.21.15.60:6789/0 conn(0x7fd98406d300 legacy=0x7fd98412bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:25.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.824+0000 7fd981d9b640 1 -- 172.21.15.60:0/1848490470 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98412b9d0 con 0x7fd984131d40 2023-08-24T10:08:25.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.824+0000 7fd981d9b640 1 --2- 172.21.15.60:0/1848490470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd984131d40 0x7fd984134160 crc :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8061ae08984cf730 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:25.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.824+0000 7fd98159a640 1 -- 172.21.15.60:0/1848490470 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd96c009dc0 con 0x7fd984131d40 2023-08-24T10:08:25.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.824+0000 7fd98159a640 1 -- 172.21.15.60:0/1848490470 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd96c00f340 con 0x7fd984131d40 2023-08-24T10:08:25.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.825+0000 7fd98159a640 1 -- 172.21.15.60:0/1848490470 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd96c00f5e0 con 0x7fd984131d40 2023-08-24T10:08:25.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.825+0000 7fd98895e640 1 -- 172.21.15.60:0/1848490470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd984131d40 msgr2=0x7fd984134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:25.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.825+0000 7fd98895e640 1 --2- 172.21.15.60:0/1848490470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd984131d40 0x7fd984134160 crc :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:25.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.825+0000 7fd98895e640 1 -- 172.21.15.60:0/1848490470 shutdown_connections 2023-08-24T10:08:25.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.825+0000 7fd98895e640 1 --2- 172.21.15.60:0/1848490470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd984131d40 0x7fd984134160 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:25.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.825+0000 7fd98895e640 1 -- 172.21.15.60:0/1848490470 >> 172.21.15.60:0/1848490470 conn(0x7fd98409e110 msgr2=0x7fd98409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:25.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.825+0000 7fd98895e640 1 -- 172.21.15.60:0/1848490470 shutdown_connections 2023-08-24T10:08:25.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.825+0000 7fd98895e640 1 -- 172.21.15.60:0/1848490470 wait complete. 2023-08-24T10:08:25.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.826+0000 7fd98895e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:25.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.827+0000 7fd98895e640 1 Processor -- start 2023-08-24T10:08:25.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.827+0000 7fd98895e640 1 -- start start 2023-08-24T10:08:25.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.827+0000 7fd98895e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd98406d300 0x7fd9841c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:25.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.827+0000 7fd98895e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd984127580 con 0x7fd98406d300 2023-08-24T10:08:25.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.827+0000 7fd98259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd98406d300 0x7fd9841c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:25.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.827+0000 7fd98259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd98406d300 0x7fd9841c2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48204/0 (socket says 172.21.15.60:48204) 2023-08-24T10:08:25.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.827+0000 7fd98259c640 1 -- 172.21.15.60:0/3441869075 learned_addr learned my addr 172.21.15.60:0/3441869075 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:25.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.828+0000 7fd98259c640 1 -- 172.21.15.60:0/3441869075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9841290d0 con 0x7fd98406d300 2023-08-24T10:08:25.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.828+0000 7fd98259c640 1 --2- 172.21.15.60:0/3441869075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd98406d300 0x7fd9841c2f20 crc :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:25.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.828+0000 7fd97affd640 1 -- 172.21.15.60:0/3441869075 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd974005ea0 con 0x7fd98406d300 2023-08-24T10:08:25.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.828+0000 7fd97affd640 1 -- 172.21.15.60:0/3441869075 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd97400eb30 con 0x7fd98406d300 2023-08-24T10:08:25.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.828+0000 7fd98895e640 1 -- 172.21.15.60:0/3441869075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9840718b0 con 0x7fd98406d300 2023-08-24T10:08:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.828+0000 7fd97affd640 1 -- 172.21.15.60:0/3441869075 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd97400edd0 con 0x7fd98406d300 2023-08-24T10:08:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.828+0000 7fd98895e640 1 -- 172.21.15.60:0/3441869075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98406f9f0 con 0x7fd98406d300 2023-08-24T10:08:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.829+0000 7fd97affd640 1 -- 172.21.15.60:0/3441869075 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fd9740070b0 con 0x7fd98406d300 2023-08-24T10:08:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.829+0000 7fd98895e640 1 -- 172.21.15.60:0/3441869075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd984061800 con 0x7fd98406d300 2023-08-24T10:08:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.829+0000 7fd97affd640 1 --2- 172.21.15.60:0/3441869075 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd964037130 0x7fd964039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:25.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.829+0000 7fd97affd640 1 -- 172.21.15.60:0/3441869075 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fd974049250 con 0x7fd98406d300 2023-08-24T10:08:25.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.829+0000 7fd981d9b640 1 --2- 172.21.15.60:0/3441869075 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd964037130 0x7fd964039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:25.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.830+0000 7fd981d9b640 1 --2- 172.21.15.60:0/3441869075 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd964037130 0x7fd964039640 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:25.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:25.834+0000 7fd97affd640 1 -- 172.21.15.60:0/3441869075 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fd974001820 con 0x7fd98406d300 2023-08-24T10:08:26.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.005+0000 7fd98895e640 1 -- 172.21.15.60:0/3441869075 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd98412a310 con 0x7fd964037130 2023-08-24T10:08:26.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.007+0000 7fd97affd640 1 -- 172.21.15.60:0/3441869075 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7fd96c009d90 con 0x7fd964037130 2023-08-24T10:08:26.008 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:26.009 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:26.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.009+0000 7fd98895e640 1 -- 172.21.15.60:0/3441869075 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd964037130 msgr2=0x7fd964039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:26.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.009+0000 7fd98895e640 1 --2- 172.21.15.60:0/3441869075 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd964037130 0x7fd964039640 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:26.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.009+0000 7fd98895e640 1 -- 172.21.15.60:0/3441869075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd98406d300 msgr2=0x7fd9841c2f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:26.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.009+0000 7fd98895e640 1 --2- 172.21.15.60:0/3441869075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd98406d300 0x7fd9841c2f20 crc :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:26.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.009+0000 7fd98895e640 1 -- 172.21.15.60:0/3441869075 shutdown_connections 2023-08-24T10:08:26.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.009+0000 7fd98895e640 1 --2- 172.21.15.60:0/3441869075 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd964037130 0x7fd964039640 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:26.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.009+0000 7fd98895e640 1 --2- 172.21.15.60:0/3441869075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd98406d300 0x7fd9841c2f20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:26.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.009+0000 7fd98895e640 1 -- 172.21.15.60:0/3441869075 >> 172.21.15.60:0/3441869075 conn(0x7fd98409e110 msgr2=0x7fd98412ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:26.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.009+0000 7fd98895e640 1 -- 172.21.15.60:0/3441869075 shutdown_connections 2023-08-24T10:08:26.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.009+0000 7fd98895e640 1 -- 172.21.15.60:0/3441869075 wait complete. 2023-08-24T10:08:26.021 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2023-08-24T10:08:25.534851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:26.022 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:08:26.023 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:26.023 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:08:26.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:26.024 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:08:26.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:26.025 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:08:26.025 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:26.025 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:08:26.026 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:26.026 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:08:26.026 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:26.026 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:08:26.027 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:26.027 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:08:26.027 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:26.027 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:08:26.028 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:26.028 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:08:26.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.135+0000 7fba98463640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:26.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.138+0000 7fba98463640 1 Processor -- start 2023-08-24T10:08:26.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.138+0000 7fba98463640 1 -- start start 2023-08-24T10:08:26.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.139+0000 7fba98463640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fba90132680 0x7fba90134aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:26.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.139+0000 7fba98463640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba9006dfc0 con 0x7fba90070c30 2023-08-24T10:08:26.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.139+0000 7fba98463640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba9006d9b0 con 0x7fba90132680 2023-08-24T10:08:26.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.139+0000 7fba961ff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fba90070c30 0x7fba90071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36876/0 (socket says 172.21.15.60:36876) 2023-08-24T10:08:26.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.139+0000 7fba961ff640 1 -- 172.21.15.60:0/3840249281 learned_addr learned my addr 172.21.15.60:0/3840249281 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:08:26.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.139+0000 7fba959fe640 1 --2- 172.21.15.60:0/3840249281 >> v2:172.21.15.60:3300/0 conn(0x7fba90132680 0x7fba90134aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:26.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.139+0000 7fba959fe640 1 -- 172.21.15.60:0/3840249281 >> v1:172.21.15.60:6789/0 conn(0x7fba90070c30 legacy=0x7fba90071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:26.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.139+0000 7fba959fe640 1 -- 172.21.15.60:0/3840249281 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9006e6e0 con 0x7fba90132680 2023-08-24T10:08:26.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.139+0000 7fba959fe640 1 --2- 172.21.15.60:0/3840249281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba90132680 0x7fba90134aa0 crc :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9b150e369c9aca server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:26.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.139+0000 7fba951fd640 1 -- 172.21.15.60:0/3840249281 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fba84009ce0 con 0x7fba90132680 2023-08-24T10:08:26.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.139+0000 7fba951fd640 1 -- 172.21.15.60:0/3840249281 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fba84009e40 con 0x7fba90132680 2023-08-24T10:08:26.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.139+0000 7fba951fd640 1 -- 172.21.15.60:0/3840249281 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fba8400e360 con 0x7fba90132680 2023-08-24T10:08:26.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.140+0000 7fba98463640 1 -- 172.21.15.60:0/3840249281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba90132680 msgr2=0x7fba90134aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:26.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.140+0000 7fba98463640 1 --2- 172.21.15.60:0/3840249281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba90132680 0x7fba90134aa0 crc :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:26.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.140+0000 7fba98463640 1 -- 172.21.15.60:0/3840249281 shutdown_connections 2023-08-24T10:08:26.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.140+0000 7fba98463640 1 --2- 172.21.15.60:0/3840249281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba90132680 0x7fba90134aa0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:26.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.140+0000 7fba98463640 1 -- 172.21.15.60:0/3840249281 >> 172.21.15.60:0/3840249281 conn(0x7fba9009e110 msgr2=0x7fba9009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:26.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.140+0000 7fba98463640 1 -- 172.21.15.60:0/3840249281 shutdown_connections 2023-08-24T10:08:26.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.140+0000 7fba98463640 1 -- 172.21.15.60:0/3840249281 wait complete. 2023-08-24T10:08:26.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.141+0000 7fba98463640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:26.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.141+0000 7fba98463640 1 Processor -- start 2023-08-24T10:08:26.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.141+0000 7fba98463640 1 -- start start 2023-08-24T10:08:26.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.142+0000 7fba98463640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba90070c30 0x7fba9012d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:26.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.142+0000 7fba98463640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9012c970 con 0x7fba90070c30 2023-08-24T10:08:26.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.142+0000 7fba961ff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba90070c30 0x7fba9012d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:26.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.142+0000 7fba961ff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba90070c30 0x7fba9012d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48226/0 (socket says 172.21.15.60:48226) 2023-08-24T10:08:26.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.142+0000 7fba961ff640 1 -- 172.21.15.60:0/551534798 learned_addr learned my addr 172.21.15.60:0/551534798 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:26.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.143+0000 7fba961ff640 1 -- 172.21.15.60:0/551534798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9012d550 con 0x7fba90070c30 2023-08-24T10:08:26.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.143+0000 7fba961ff640 1 --2- 172.21.15.60:0/551534798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba90070c30 0x7fba9012d890 crc :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:26.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.143+0000 7fba8effd640 1 -- 172.21.15.60:0/551534798 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fba8000f570 con 0x7fba90070c30 2023-08-24T10:08:26.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.143+0000 7fba8effd640 1 -- 172.21.15.60:0/551534798 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fba8000f6d0 con 0x7fba90070c30 2023-08-24T10:08:26.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.143+0000 7fba98463640 1 -- 172.21.15.60:0/551534798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9012cba0 con 0x7fba90070c30 2023-08-24T10:08:26.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.143+0000 7fba8effd640 1 -- 172.21.15.60:0/551534798 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fba8000f970 con 0x7fba90070c30 2023-08-24T10:08:26.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.143+0000 7fba98463640 1 -- 172.21.15.60:0/551534798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9012dfb0 con 0x7fba90070c30 2023-08-24T10:08:26.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.144+0000 7fba8effd640 1 -- 172.21.15.60:0/551534798 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fba8000fb60 con 0x7fba90070c30 2023-08-24T10:08:26.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.144+0000 7fba98463640 1 -- 172.21.15.60:0/551534798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba90061800 con 0x7fba90070c30 2023-08-24T10:08:26.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.144+0000 7fba8effd640 1 --2- 172.21.15.60:0/551534798 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fba7c037130 0x7fba7c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:26.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.144+0000 7fba8effd640 1 -- 172.21.15.60:0/551534798 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fba80049cc0 con 0x7fba90070c30 2023-08-24T10:08:26.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.144+0000 7fba959fe640 1 --2- 172.21.15.60:0/551534798 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fba7c037130 0x7fba7c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:26.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.145+0000 7fba959fe640 1 --2- 172.21.15.60:0/551534798 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fba7c037130 0x7fba7c039640 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:26.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.149+0000 7fba8effd640 1 -- 172.21.15.60:0/551534798 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fba8000a460 con 0x7fba90070c30 2023-08-24T10:08:26.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.395+0000 7fba98463640 1 -- 172.21.15.60:0/551534798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fba900706e0 con 0x7fba90070c30 2023-08-24T10:08:26.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.395+0000 7fba8effd640 1 -- 172.21.15.60:0/551534798 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7fba8000be60 con 0x7fba90070c30 2023-08-24T10:08:26.397 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:26.397 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":217,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:08:26.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.398+0000 7fba98463640 1 -- 172.21.15.60:0/551534798 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fba7c037130 msgr2=0x7fba7c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:26.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.398+0000 7fba98463640 1 --2- 172.21.15.60:0/551534798 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fba7c037130 0x7fba7c039640 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:26.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.398+0000 7fba98463640 1 -- 172.21.15.60:0/551534798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba90070c30 msgr2=0x7fba9012d890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:26.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.398+0000 7fba98463640 1 --2- 172.21.15.60:0/551534798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba90070c30 0x7fba9012d890 crc :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:26.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.398+0000 7fba98463640 1 -- 172.21.15.60:0/551534798 shutdown_connections 2023-08-24T10:08:26.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.398+0000 7fba98463640 1 --2- 172.21.15.60:0/551534798 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fba7c037130 0x7fba7c039640 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:26.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.398+0000 7fba98463640 1 --2- 172.21.15.60:0/551534798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba90070c30 0x7fba9012d890 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:26.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.398+0000 7fba98463640 1 -- 172.21.15.60:0/551534798 >> 172.21.15.60:0/551534798 conn(0x7fba9009e110 msgr2=0x7fba901b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:26.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.398+0000 7fba98463640 1 -- 172.21.15.60:0/551534798 shutdown_connections 2023-08-24T10:08:26.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.398+0000 7fba98463640 1 -- 172.21.15.60:0/551534798 wait complete. 2023-08-24T10:08:26.410 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:08:26.411 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:26.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.517+0000 7feb4826c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:26.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.521+0000 7feb4826c640 1 Processor -- start 2023-08-24T10:08:26.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.521+0000 7feb4826c640 1 -- start start 2023-08-24T10:08:26.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.522+0000 7feb4826c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feb40132b50 0x7feb40132f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:26.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.522+0000 7feb4826c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb40132610 con 0x7feb4012f110 2023-08-24T10:08:26.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.522+0000 7feb4826c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb4009c260 con 0x7feb40132b50 2023-08-24T10:08:26.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.522+0000 7feb46008640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feb40132b50 0x7feb40132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.522+0000 7feb46008640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feb40132b50 0x7feb40132f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48234/0 (socket says 172.21.15.60:48234) 2023-08-24T10:08:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.522+0000 7feb46008640 1 -- 172.21.15.60:0/800946147 learned_addr learned my addr 172.21.15.60:0/800946147 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.522+0000 7feb45807640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7feb4012f110 0x7feb4012f520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36892/0 (socket says 172.21.15.60:36892) 2023-08-24T10:08:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.523+0000 7feb46008640 1 -- 172.21.15.60:0/800946147 >> v1:172.21.15.60:6789/0 conn(0x7feb4012f110 legacy=0x7feb4012f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.523+0000 7feb46008640 1 -- 172.21.15.60:0/800946147 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb40132840 con 0x7feb40132b50 2023-08-24T10:08:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.523+0000 7feb46008640 1 --2- 172.21.15.60:0/800946147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40132b50 0x7feb40132f60 crc :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431fb00a9e3f8dfc server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.523+0000 7feb45006640 1 -- 172.21.15.60:0/800946147 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7feb38009dd0 con 0x7feb40132b50 2023-08-24T10:08:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.523+0000 7feb45006640 1 -- 172.21.15.60:0/800946147 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7feb3800e230 con 0x7feb40132b50 2023-08-24T10:08:26.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.523+0000 7feb45006640 1 -- 172.21.15.60:0/800946147 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7feb3800e4d0 con 0x7feb40132b50 2023-08-24T10:08:26.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.524+0000 7feb4826c640 1 -- 172.21.15.60:0/800946147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40132b50 msgr2=0x7feb40132f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:26.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.524+0000 7feb4826c640 1 --2- 172.21.15.60:0/800946147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40132b50 0x7feb40132f60 crc :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:26.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.524+0000 7feb4826c640 1 -- 172.21.15.60:0/800946147 shutdown_connections 2023-08-24T10:08:26.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.524+0000 7feb4826c640 1 --2- 172.21.15.60:0/800946147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40132b50 0x7feb40132f60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:26.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.524+0000 7feb4826c640 1 -- 172.21.15.60:0/800946147 >> 172.21.15.60:0/800946147 conn(0x7feb400a26a0 msgr2=0x7feb400a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:26.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.524+0000 7feb4826c640 1 -- 172.21.15.60:0/800946147 shutdown_connections 2023-08-24T10:08:26.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.524+0000 7feb4826c640 1 -- 172.21.15.60:0/800946147 wait complete. 2023-08-24T10:08:26.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.524+0000 7feb4826c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:26.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.525+0000 7feb4826c640 1 Processor -- start 2023-08-24T10:08:26.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.525+0000 7feb4826c640 1 -- start start 2023-08-24T10:08:26.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.525+0000 7feb4826c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb4012f110 0x7feb4009fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:26.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.525+0000 7feb4826c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4012cc80 con 0x7feb4012f110 2023-08-24T10:08:26.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.526+0000 7feb46008640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb4012f110 0x7feb4009fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:26.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.526+0000 7feb46008640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb4012f110 0x7feb4009fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48248/0 (socket says 172.21.15.60:48248) 2023-08-24T10:08:26.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.526+0000 7feb46008640 1 -- 172.21.15.60:0/2047829252 learned_addr learned my addr 172.21.15.60:0/2047829252 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:26.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.526+0000 7feb46008640 1 -- 172.21.15.60:0/2047829252 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4009f400 con 0x7feb4012f110 2023-08-24T10:08:26.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.526+0000 7feb46008640 1 --2- 172.21.15.60:0/2047829252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb4012f110 0x7feb4009fcd0 crc :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:26.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.526+0000 7feb3effd640 1 -- 172.21.15.60:0/2047829252 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7feb38014540 con 0x7feb4012f110 2023-08-24T10:08:26.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.526+0000 7feb3effd640 1 -- 172.21.15.60:0/2047829252 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7feb380146a0 con 0x7feb4012f110 2023-08-24T10:08:26.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.526+0000 7feb4826c640 1 -- 172.21.15.60:0/2047829252 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb400a18e0 con 0x7feb4012f110 2023-08-24T10:08:26.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.526+0000 7feb3effd640 1 -- 172.21.15.60:0/2047829252 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7feb38014940 con 0x7feb4012f110 2023-08-24T10:08:26.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.526+0000 7feb4826c640 1 -- 172.21.15.60:0/2047829252 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4006a1d0 con 0x7feb4012f110 2023-08-24T10:08:26.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.527+0000 7feb3effd640 1 -- 172.21.15.60:0/2047829252 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7feb38014af0 con 0x7feb4012f110 2023-08-24T10:08:26.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.527+0000 7feb4826c640 1 -- 172.21.15.60:0/2047829252 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb4011c2b0 con 0x7feb4012f110 2023-08-24T10:08:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.527+0000 7feb3effd640 1 --2- 172.21.15.60:0/2047829252 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7feb2c037130 0x7feb2c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.527+0000 7feb3effd640 1 -- 172.21.15.60:0/2047829252 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7feb38019040 con 0x7feb4012f110 2023-08-24T10:08:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.528+0000 7feb45807640 1 --2- 172.21.15.60:0/2047829252 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7feb2c037130 0x7feb2c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.528+0000 7feb45807640 1 --2- 172.21.15.60:0/2047829252 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7feb2c037130 0x7feb2c039640 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.532+0000 7feb3effd640 1 -- 172.21.15.60:0/2047829252 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7feb38018190 con 0x7feb4012f110 2023-08-24T10:08:26.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.704+0000 7feb4826c640 1 -- 172.21.15.60:0/2047829252 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feb401334a0 con 0x7feb2c037130 2023-08-24T10:08:26.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.705+0000 7feb3effd640 1 -- 172.21.15.60:0/2047829252 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7feb340011a0 con 0x7feb2c037130 2023-08-24T10:08:26.707 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:26.707 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:26.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.707+0000 7feb4826c640 1 -- 172.21.15.60:0/2047829252 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7feb2c037130 msgr2=0x7feb2c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:26.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.707+0000 7feb4826c640 1 --2- 172.21.15.60:0/2047829252 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7feb2c037130 0x7feb2c039640 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:26.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.707+0000 7feb4826c640 1 -- 172.21.15.60:0/2047829252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb4012f110 msgr2=0x7feb4009fcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:26.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.707+0000 7feb4826c640 1 --2- 172.21.15.60:0/2047829252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb4012f110 0x7feb4009fcd0 crc :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:26.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.707+0000 7feb4826c640 1 -- 172.21.15.60:0/2047829252 shutdown_connections 2023-08-24T10:08:26.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.707+0000 7feb4826c640 1 --2- 172.21.15.60:0/2047829252 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7feb2c037130 0x7feb2c039640 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:26.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.707+0000 7feb4826c640 1 --2- 172.21.15.60:0/2047829252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb4012f110 0x7feb4009fcd0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:26.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.707+0000 7feb4826c640 1 -- 172.21.15.60:0/2047829252 >> 172.21.15.60:0/2047829252 conn(0x7feb400a26a0 msgr2=0x7feb401ba920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:26.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.707+0000 7feb4826c640 1 -- 172.21.15.60:0/2047829252 shutdown_connections 2023-08-24T10:08:26.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:26.707+0000 7feb4826c640 1 -- 172.21.15.60:0/2047829252 wait complete. 2023-08-24T10:08:26.720 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2023-08-24T10:08:25.534851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:29.723 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:29.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.844+0000 7efe97b9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.848+0000 7efe97b9e640 1 Processor -- start 2023-08-24T10:08:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.848+0000 7efe97b9e640 1 -- start start 2023-08-24T10:08:29.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.849+0000 7efe97b9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efe90131d40 0x7efe90134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:29.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.849+0000 7efe97b9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe90099440 con 0x7efe9006d300 2023-08-24T10:08:29.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.849+0000 7efe97b9e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9009a050 con 0x7efe90131d40 2023-08-24T10:08:29.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.849+0000 7efe95139640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efe90131d40 0x7efe90134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:29.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.849+0000 7efe95139640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efe90131d40 0x7efe90134160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37560/0 (socket says 172.21.15.60:37560) 2023-08-24T10:08:29.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.849+0000 7efe95139640 1 -- 172.21.15.60:0/3893573787 learned_addr learned my addr 172.21.15.60:0/3893573787 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:29.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.849+0000 7efe9593a640 1 --1- 172.21.15.60:0/3893573787 >> v1:172.21.15.60:6789/0 conn(0x7efe9006d300 0x7efe9012bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34406/0 (socket says 172.21.15.60:34406) 2023-08-24T10:08:29.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.849+0000 7efe95139640 1 -- 172.21.15.60:0/3893573787 >> v1:172.21.15.60:6789/0 conn(0x7efe9006d300 legacy=0x7efe9012bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:29.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.849+0000 7efe95139640 1 -- 172.21.15.60:0/3893573787 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9012b9d0 con 0x7efe90131d40 2023-08-24T10:08:29.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.850+0000 7efe95139640 1 --2- 172.21.15.60:0/3893573787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe90131d40 0x7efe90134160 crc :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543d479a355c1f2b server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:29.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.850+0000 7efe94938640 1 -- 172.21.15.60:0/3893573787 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efe80010570 con 0x7efe90131d40 2023-08-24T10:08:29.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.850+0000 7efe94938640 1 -- 172.21.15.60:0/3893573787 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7efe800106d0 con 0x7efe90131d40 2023-08-24T10:08:29.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.850+0000 7efe94938640 1 -- 172.21.15.60:0/3893573787 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efe80010970 con 0x7efe90131d40 2023-08-24T10:08:29.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.850+0000 7efe97b9e640 1 -- 172.21.15.60:0/3893573787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe90131d40 msgr2=0x7efe90134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:29.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.850+0000 7efe97b9e640 1 --2- 172.21.15.60:0/3893573787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe90131d40 0x7efe90134160 crc :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:29.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.850+0000 7efe97b9e640 1 -- 172.21.15.60:0/3893573787 shutdown_connections 2023-08-24T10:08:29.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.850+0000 7efe97b9e640 1 --2- 172.21.15.60:0/3893573787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe90131d40 0x7efe90134160 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:29.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.850+0000 7efe97b9e640 1 -- 172.21.15.60:0/3893573787 >> 172.21.15.60:0/3893573787 conn(0x7efe9009e110 msgr2=0x7efe9009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:29.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.851+0000 7efe97b9e640 1 -- 172.21.15.60:0/3893573787 shutdown_connections 2023-08-24T10:08:29.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.851+0000 7efe97b9e640 1 -- 172.21.15.60:0/3893573787 wait complete. 2023-08-24T10:08:29.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.851+0000 7efe97b9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:29.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.852+0000 7efe97b9e640 1 Processor -- start 2023-08-24T10:08:29.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.852+0000 7efe97b9e640 1 -- start start 2023-08-24T10:08:29.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.852+0000 7efe97b9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe9006d300 0x7efe901c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:29.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.852+0000 7efe97b9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe90127640 con 0x7efe9006d300 2023-08-24T10:08:29.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.853+0000 7efe9593a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe9006d300 0x7efe901c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:29.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.853+0000 7efe9593a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe9006d300 0x7efe901c2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37572/0 (socket says 172.21.15.60:37572) 2023-08-24T10:08:29.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.853+0000 7efe9593a640 1 -- 172.21.15.60:0/3804112873 learned_addr learned my addr 172.21.15.60:0/3804112873 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:29.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.853+0000 7efe9593a640 1 -- 172.21.15.60:0/3804112873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9006ffd0 con 0x7efe9006d300 2023-08-24T10:08:29.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.853+0000 7efe9593a640 1 --2- 172.21.15.60:0/3804112873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe9006d300 0x7efe901c2f20 crc :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:29.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.853+0000 7efe867fc640 1 -- 172.21.15.60:0/3804112873 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efe8c0071e0 con 0x7efe9006d300 2023-08-24T10:08:29.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.853+0000 7efe867fc640 1 -- 172.21.15.60:0/3804112873 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7efe8c007340 con 0x7efe9006d300 2023-08-24T10:08:29.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.853+0000 7efe97b9e640 1 -- 172.21.15.60:0/3804112873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe90071980 con 0x7efe9006d300 2023-08-24T10:08:29.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.853+0000 7efe867fc640 1 -- 172.21.15.60:0/3804112873 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efe8c0075e0 con 0x7efe9006d300 2023-08-24T10:08:29.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.853+0000 7efe97b9e640 1 -- 172.21.15.60:0/3804112873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9006f330 con 0x7efe9006d300 2023-08-24T10:08:29.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.854+0000 7efe867fc640 1 -- 172.21.15.60:0/3804112873 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7efe8c0057e0 con 0x7efe9006d300 2023-08-24T10:08:29.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.854+0000 7efe97b9e640 1 -- 172.21.15.60:0/3804112873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe90061800 con 0x7efe9006d300 2023-08-24T10:08:29.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.854+0000 7efe867fc640 1 --2- 172.21.15.60:0/3804112873 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efe74037130 0x7efe74039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:29.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.854+0000 7efe867fc640 1 -- 172.21.15.60:0/3804112873 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7efe8c046330 con 0x7efe9006d300 2023-08-24T10:08:29.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.855+0000 7efe95139640 1 --2- 172.21.15.60:0/3804112873 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efe74037130 0x7efe74039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:29.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.855+0000 7efe95139640 1 --2- 172.21.15.60:0/3804112873 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efe74037130 0x7efe74039640 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:29.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:29.859+0000 7efe867fc640 1 -- 172.21.15.60:0/3804112873 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7efe8c00d330 con 0x7efe9006d300 2023-08-24T10:08:30.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.034+0000 7efe97b9e640 1 -- 172.21.15.60:0/3804112873 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efe9006da20 con 0x7efe74037130 2023-08-24T10:08:30.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.035+0000 7efe867fc640 1 -- 172.21.15.60:0/3804112873 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7efe80010440 con 0x7efe74037130 2023-08-24T10:08:30.037 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:30.037 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:30.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.037+0000 7efe97b9e640 1 -- 172.21.15.60:0/3804112873 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efe74037130 msgr2=0x7efe74039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:30.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.037+0000 7efe97b9e640 1 --2- 172.21.15.60:0/3804112873 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efe74037130 0x7efe74039640 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:30.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.037+0000 7efe97b9e640 1 -- 172.21.15.60:0/3804112873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe9006d300 msgr2=0x7efe901c2f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:30.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.037+0000 7efe97b9e640 1 --2- 172.21.15.60:0/3804112873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe9006d300 0x7efe901c2f20 crc :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:30.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.037+0000 7efe97b9e640 1 -- 172.21.15.60:0/3804112873 shutdown_connections 2023-08-24T10:08:30.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.037+0000 7efe97b9e640 1 --2- 172.21.15.60:0/3804112873 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efe74037130 0x7efe74039640 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:30.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.037+0000 7efe97b9e640 1 --2- 172.21.15.60:0/3804112873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe9006d300 0x7efe901c2f20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:30.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.037+0000 7efe97b9e640 1 -- 172.21.15.60:0/3804112873 >> 172.21.15.60:0/3804112873 conn(0x7efe9009e110 msgr2=0x7efe9012ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:30.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.038+0000 7efe97b9e640 1 -- 172.21.15.60:0/3804112873 shutdown_connections 2023-08-24T10:08:30.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.038+0000 7efe97b9e640 1 -- 172.21.15.60:0/3804112873 wait complete. 2023-08-24T10:08:30.049 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2023-08-24T10:08:29.535441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:30.050 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:30.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.157+0000 7f399ecec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.161+0000 7f399ecec640 1 Processor -- start 2023-08-24T10:08:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.161+0000 7f399ecec640 1 -- start start 2023-08-24T10:08:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.162+0000 7f399ecec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3998132b50 0x7f3998132f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.162+0000 7f399ecec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3998132610 con 0x7f399812f110 2023-08-24T10:08:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.162+0000 7f399ecec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f399809c260 con 0x7f3998132b50 2023-08-24T10:08:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.162+0000 7f399ca88640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3998132b50 0x7f3998132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.162+0000 7f399ca88640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3998132b50 0x7f3998132f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37574/0 (socket says 172.21.15.60:37574) 2023-08-24T10:08:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.162+0000 7f399ca88640 1 -- 172.21.15.60:0/562327797 learned_addr learned my addr 172.21.15.60:0/562327797 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.162+0000 7f3997fff640 1 --1- 172.21.15.60:0/562327797 >> v1:172.21.15.60:6789/0 conn(0x7f399812f110 0x7f399812f520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34418/0 (socket says 172.21.15.60:34418) 2023-08-24T10:08:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.162+0000 7f399ca88640 1 -- 172.21.15.60:0/562327797 >> v1:172.21.15.60:6789/0 conn(0x7f399812f110 legacy=0x7f399812f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:30.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.162+0000 7f399ca88640 1 -- 172.21.15.60:0/562327797 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3998132840 con 0x7f3998132b50 2023-08-24T10:08:30.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.162+0000 7f399ca88640 1 --2- 172.21.15.60:0/562327797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3998132b50 0x7f3998132f60 crc :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b07a2087833f07 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:30.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.162+0000 7f39977fe640 1 -- 172.21.15.60:0/562327797 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f399000e040 con 0x7f3998132b50 2023-08-24T10:08:30.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.162+0000 7f39977fe640 1 -- 172.21.15.60:0/562327797 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f39900091e0 con 0x7f3998132b50 2023-08-24T10:08:30.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.162+0000 7f39977fe640 1 -- 172.21.15.60:0/562327797 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3990007730 con 0x7f3998132b50 2023-08-24T10:08:30.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.163+0000 7f399ecec640 1 -- 172.21.15.60:0/562327797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3998132b50 msgr2=0x7f3998132f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:30.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.163+0000 7f399ecec640 1 --2- 172.21.15.60:0/562327797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3998132b50 0x7f3998132f60 crc :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:30.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.163+0000 7f399ecec640 1 -- 172.21.15.60:0/562327797 shutdown_connections 2023-08-24T10:08:30.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.163+0000 7f399ecec640 1 --2- 172.21.15.60:0/562327797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3998132b50 0x7f3998132f60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:30.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.163+0000 7f399ecec640 1 -- 172.21.15.60:0/562327797 >> 172.21.15.60:0/562327797 conn(0x7f39980a26a0 msgr2=0x7f39980a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:30.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.163+0000 7f399ecec640 1 -- 172.21.15.60:0/562327797 shutdown_connections 2023-08-24T10:08:30.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.163+0000 7f399ecec640 1 -- 172.21.15.60:0/562327797 wait complete. 2023-08-24T10:08:30.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.163+0000 7f399ecec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:30.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.164+0000 7f399ecec640 1 Processor -- start 2023-08-24T10:08:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.164+0000 7f399ecec640 1 -- start start 2023-08-24T10:08:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.165+0000 7f399ecec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399812f110 0x7f399809fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.165+0000 7f399ecec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399812cc80 con 0x7f399812f110 2023-08-24T10:08:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.165+0000 7f399ca88640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399812f110 0x7f399809fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.165+0000 7f399ca88640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399812f110 0x7f399809fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37578/0 (socket says 172.21.15.60:37578) 2023-08-24T10:08:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.165+0000 7f399ca88640 1 -- 172.21.15.60:0/2898153417 learned_addr learned my addr 172.21.15.60:0/2898153417 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.166+0000 7f399ca88640 1 -- 172.21.15.60:0/2898153417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399809f400 con 0x7f399812f110 2023-08-24T10:08:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.166+0000 7f399ca88640 1 --2- 172.21.15.60:0/2898153417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399812f110 0x7f399809fcd0 crc :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:30.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.166+0000 7f39957fa640 1 -- 172.21.15.60:0/2898153417 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f399000e040 con 0x7f399812f110 2023-08-24T10:08:30.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.166+0000 7f39957fa640 1 -- 172.21.15.60:0/2898153417 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3990010800 con 0x7f399812f110 2023-08-24T10:08:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.166+0000 7f39957fa640 1 -- 172.21.15.60:0/2898153417 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3990010aa0 con 0x7f399812f110 2023-08-24T10:08:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.166+0000 7f399ecec640 1 -- 172.21.15.60:0/2898153417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39980a18e0 con 0x7f399812f110 2023-08-24T10:08:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.166+0000 7f399ecec640 1 -- 172.21.15.60:0/2898153417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399806a1d0 con 0x7f399812f110 2023-08-24T10:08:30.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.167+0000 7f39957fa640 1 -- 172.21.15.60:0/2898153417 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f3990010c00 con 0x7f399812f110 2023-08-24T10:08:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.167+0000 7f399ecec640 1 -- 172.21.15.60:0/2898153417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399811c2b0 con 0x7f399812f110 2023-08-24T10:08:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.167+0000 7f39957fa640 1 --2- 172.21.15.60:0/2898153417 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3978037130 0x7f3978039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.167+0000 7f39957fa640 1 -- 172.21.15.60:0/2898153417 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f3990047a20 con 0x7f399812f110 2023-08-24T10:08:30.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.167+0000 7f3997fff640 1 --2- 172.21.15.60:0/2898153417 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3978037130 0x7f3978039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:30.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.168+0000 7f3997fff640 1 --2- 172.21.15.60:0/2898153417 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3978037130 0x7f3978039640 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.172+0000 7f39957fa640 1 -- 172.21.15.60:0/2898153417 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f399000e1a0 con 0x7f399812f110 2023-08-24T10:08:30.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.348+0000 7f399ecec640 1 -- 172.21.15.60:0/2898153417 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f39981334a0 con 0x7f3978037130 2023-08-24T10:08:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.349+0000 7f39957fa640 1 -- 172.21.15.60:0/2898153417 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f39800070b0 con 0x7f3978037130 2023-08-24T10:08:30.350 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:30.351 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:30.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.351+0000 7f399ecec640 1 -- 172.21.15.60:0/2898153417 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3978037130 msgr2=0x7f3978039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:30.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.351+0000 7f399ecec640 1 --2- 172.21.15.60:0/2898153417 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3978037130 0x7f3978039640 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:30.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.351+0000 7f399ecec640 1 -- 172.21.15.60:0/2898153417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399812f110 msgr2=0x7f399809fcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:30.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.351+0000 7f399ecec640 1 --2- 172.21.15.60:0/2898153417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399812f110 0x7f399809fcd0 crc :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:30.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.351+0000 7f399ecec640 1 -- 172.21.15.60:0/2898153417 shutdown_connections 2023-08-24T10:08:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.351+0000 7f399ecec640 1 --2- 172.21.15.60:0/2898153417 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3978037130 0x7f3978039640 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.351+0000 7f399ecec640 1 --2- 172.21.15.60:0/2898153417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399812f110 0x7f399809fcd0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.351+0000 7f399ecec640 1 -- 172.21.15.60:0/2898153417 >> 172.21.15.60:0/2898153417 conn(0x7f39980a26a0 msgr2=0x7f39981ba920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.351+0000 7f399ecec640 1 -- 172.21.15.60:0/2898153417 shutdown_connections 2023-08-24T10:08:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.351+0000 7f399ecec640 1 -- 172.21.15.60:0/2898153417 wait complete. 2023-08-24T10:08:30.365 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2023-08-24T10:08:29.535441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:30.366 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:08:30.366 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:30.367 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:08:30.367 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:30.367 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:08:30.368 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:30.368 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:08:30.368 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:30.368 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:08:30.369 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:30.369 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:08:30.370 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:30.371 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:08:30.371 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:30.372 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:08:30.372 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:30.372 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:08:30.372 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:30.373 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:08:30.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~203s 2023-08-24T10:08:30.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~203s 2023-08-24T10:08:30.398 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~203s 2023-08-24T10:08:30.398 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~203s 2023-08-24T10:08:30.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.480+0000 7fbf9dcec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:30.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.483+0000 7fbf9dcec640 1 Processor -- start 2023-08-24T10:08:30.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.483+0000 7fbf9dcec640 1 -- start start 2023-08-24T10:08:30.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.484+0000 7fbf9dcec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbf9812ece0 0x7fbf9812f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:30.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.484+0000 7fbf9dcec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf9809a650 con 0x7fbf9812f630 2023-08-24T10:08:30.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.484+0000 7fbf9dcec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9806da00 con 0x7fbf9812ece0 2023-08-24T10:08:30.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.484+0000 7fbf977fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbf9812ece0 0x7fbf9812f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:30.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.484+0000 7fbf977fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbf9812ece0 0x7fbf9812f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37590/0 (socket says 172.21.15.60:37590) 2023-08-24T10:08:30.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.484+0000 7fbf96ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbf9812f630 0x7fbf98097fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34428/0 (socket says 172.21.15.60:34428) 2023-08-24T10:08:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.484+0000 7fbf977fe640 1 -- 172.21.15.60:0/735273612 learned_addr learned my addr 172.21.15.60:0/735273612 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.484+0000 7fbf977fe640 1 -- 172.21.15.60:0/735273612 >> v1:172.21.15.60:6789/0 conn(0x7fbf9812f630 legacy=0x7fbf98097fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.484+0000 7fbf977fe640 1 -- 172.21.15.60:0/735273612 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9812ea10 con 0x7fbf9812ece0 2023-08-24T10:08:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.484+0000 7fbf977fe640 1 --2- 172.21.15.60:0/735273612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9812ece0 0x7fbf9812f0f0 crc :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744b8449c287daed server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.485+0000 7fbf967fc640 1 -- 172.21.15.60:0/735273612 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbf88009dd0 con 0x7fbf9812ece0 2023-08-24T10:08:30.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.485+0000 7fbf967fc640 1 -- 172.21.15.60:0/735273612 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fbf8800e230 con 0x7fbf9812ece0 2023-08-24T10:08:30.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.485+0000 7fbf967fc640 1 -- 172.21.15.60:0/735273612 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbf8800e4d0 con 0x7fbf9812ece0 2023-08-24T10:08:30.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.485+0000 7fbf9dcec640 1 -- 172.21.15.60:0/735273612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9812ece0 msgr2=0x7fbf9812f0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:30.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.485+0000 7fbf9dcec640 1 --2- 172.21.15.60:0/735273612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9812ece0 0x7fbf9812f0f0 crc :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:30.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.485+0000 7fbf9dcec640 1 -- 172.21.15.60:0/735273612 shutdown_connections 2023-08-24T10:08:30.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.485+0000 7fbf9dcec640 1 --2- 172.21.15.60:0/735273612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9812ece0 0x7fbf9812f0f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:30.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.485+0000 7fbf9dcec640 1 -- 172.21.15.60:0/735273612 >> 172.21.15.60:0/735273612 conn(0x7fbf9809e110 msgr2=0x7fbf9809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:30.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.485+0000 7fbf9dcec640 1 -- 172.21.15.60:0/735273612 shutdown_connections 2023-08-24T10:08:30.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.485+0000 7fbf9dcec640 1 -- 172.21.15.60:0/735273612 wait complete. 2023-08-24T10:08:30.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.485+0000 7fbf9dcec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:30.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.486+0000 7fbf9dcec640 1 Processor -- start 2023-08-24T10:08:30.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.487+0000 7fbf9dcec640 1 -- start start 2023-08-24T10:08:30.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.487+0000 7fbf9dcec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9812ece0 0x7fbf98135f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:30.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.487+0000 7fbf9dcec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf98125a60 con 0x7fbf9812ece0 2023-08-24T10:08:30.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.487+0000 7fbf977fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9812ece0 0x7fbf98135f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:30.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.487+0000 7fbf977fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9812ece0 0x7fbf98135f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37600/0 (socket says 172.21.15.60:37600) 2023-08-24T10:08:30.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.487+0000 7fbf977fe640 1 -- 172.21.15.60:0/3025816240 learned_addr learned my addr 172.21.15.60:0/3025816240 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:30.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.488+0000 7fbf977fe640 1 -- 172.21.15.60:0/3025816240 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf98135a60 con 0x7fbf9812ece0 2023-08-24T10:08:30.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.488+0000 7fbf977fe640 1 --2- 172.21.15.60:0/3025816240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9812ece0 0x7fbf98135f40 crc :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:30.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.488+0000 7fbf7ffff640 1 -- 172.21.15.60:0/3025816240 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbf88014410 con 0x7fbf9812ece0 2023-08-24T10:08:30.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.488+0000 7fbf7ffff640 1 -- 172.21.15.60:0/3025816240 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fbf88014570 con 0x7fbf9812ece0 2023-08-24T10:08:30.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.488+0000 7fbf9dcec640 1 -- 172.21.15.60:0/3025816240 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf98125ee0 con 0x7fbf9812ece0 2023-08-24T10:08:30.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.488+0000 7fbf7ffff640 1 -- 172.21.15.60:0/3025816240 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbf88014810 con 0x7fbf9812ece0 2023-08-24T10:08:30.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.488+0000 7fbf9dcec640 1 -- 172.21.15.60:0/3025816240 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf98127a80 con 0x7fbf9812ece0 2023-08-24T10:08:30.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.489+0000 7fbf7ffff640 1 -- 172.21.15.60:0/3025816240 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fbf880149c0 con 0x7fbf9812ece0 2023-08-24T10:08:30.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.489+0000 7fbf9dcec640 1 -- 172.21.15.60:0/3025816240 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf981255b0 con 0x7fbf9812ece0 2023-08-24T10:08:30.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.489+0000 7fbf7ffff640 1 --2- 172.21.15.60:0/3025816240 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fbf84037130 0x7fbf84039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:30.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.489+0000 7fbf7ffff640 1 -- 172.21.15.60:0/3025816240 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fbf88019040 con 0x7fbf9812ece0 2023-08-24T10:08:30.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.489+0000 7fbf96ffd640 1 --2- 172.21.15.60:0/3025816240 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fbf84037130 0x7fbf84039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:30.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.490+0000 7fbf96ffd640 1 --2- 172.21.15.60:0/3025816240 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fbf84037130 0x7fbf84039640 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:30.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.493+0000 7fbf7ffff640 1 -- 172.21.15.60:0/3025816240 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fbf8800f8f0 con 0x7fbf9812ece0 2023-08-24T10:08:30.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.742+0000 7fbf9dcec640 1 -- 172.21.15.60:0/3025816240 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fbf98135860 con 0x7fbf9812ece0 2023-08-24T10:08:30.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.742+0000 7fbf7ffff640 1 -- 172.21.15.60:0/3025816240 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7fbf88005960 con 0x7fbf9812ece0 2023-08-24T10:08:30.744 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:30.744 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":221,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:08:30.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.744+0000 7fbf9dcec640 1 -- 172.21.15.60:0/3025816240 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fbf84037130 msgr2=0x7fbf84039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:30.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.744+0000 7fbf9dcec640 1 --2- 172.21.15.60:0/3025816240 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fbf84037130 0x7fbf84039640 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:30.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.744+0000 7fbf9dcec640 1 -- 172.21.15.60:0/3025816240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9812ece0 msgr2=0x7fbf98135f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:30.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.744+0000 7fbf9dcec640 1 --2- 172.21.15.60:0/3025816240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9812ece0 0x7fbf98135f40 crc :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:30.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.745+0000 7fbf9dcec640 1 -- 172.21.15.60:0/3025816240 shutdown_connections 2023-08-24T10:08:30.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.745+0000 7fbf9dcec640 1 --2- 172.21.15.60:0/3025816240 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fbf84037130 0x7fbf84039640 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:30.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.745+0000 7fbf9dcec640 1 --2- 172.21.15.60:0/3025816240 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf9812ece0 0x7fbf98135f40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:30.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.745+0000 7fbf9dcec640 1 -- 172.21.15.60:0/3025816240 >> 172.21.15.60:0/3025816240 conn(0x7fbf9809e110 msgr2=0x7fbf98097710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:30.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.745+0000 7fbf9dcec640 1 -- 172.21.15.60:0/3025816240 shutdown_connections 2023-08-24T10:08:30.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.745+0000 7fbf9dcec640 1 -- 172.21.15.60:0/3025816240 wait complete. 2023-08-24T10:08:30.756 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:08:30.757 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.864+0000 7fb351230640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.867+0000 7fb351230640 1 Processor -- start 2023-08-24T10:08:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.868+0000 7fb351230640 1 -- start start 2023-08-24T10:08:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.868+0000 7fb351230640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb34c06d300 0x7fb34c12bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:30.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.868+0000 7fb351230640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb34c099440 con 0x7fb34c12c1c0 2023-08-24T10:08:30.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.868+0000 7fb351230640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34c09a050 con 0x7fb34c06d300 2023-08-24T10:08:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.868+0000 7fb34ad9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb34c06d300 0x7fb34c12bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.868+0000 7fb34ad9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb34c06d300 0x7fb34c12bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37602/0 (socket says 172.21.15.60:37602) 2023-08-24T10:08:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.868+0000 7fb34ad9d640 1 -- 172.21.15.60:0/3410508024 learned_addr learned my addr 172.21.15.60:0/3410508024 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.869+0000 7fb34ad9d640 1 -- 172.21.15.60:0/3410508024 >> v1:172.21.15.60:6789/0 conn(0x7fb34c12c1c0 legacy=0x7fb34c12ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.869+0000 7fb34ad9d640 1 -- 172.21.15.60:0/3410508024 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c12b9d0 con 0x7fb34c06d300 2023-08-24T10:08:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.869+0000 7fb34ad9d640 1 --2- 172.21.15.60:0/3410508024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb34c06d300 0x7fb34c12bc80 crc :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb8783c30dd3d5f server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.869+0000 7fb349d9b640 1 -- 172.21.15.60:0/3410508024 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb33c009dd0 con 0x7fb34c06d300 2023-08-24T10:08:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.869+0000 7fb349d9b640 1 -- 172.21.15.60:0/3410508024 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb33c00f340 con 0x7fb34c06d300 2023-08-24T10:08:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.869+0000 7fb349d9b640 1 -- 172.21.15.60:0/3410508024 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb33c00f5e0 con 0x7fb34c06d300 2023-08-24T10:08:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.870+0000 7fb351230640 1 -- 172.21.15.60:0/3410508024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb34c06d300 msgr2=0x7fb34c12bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.870+0000 7fb351230640 1 --2- 172.21.15.60:0/3410508024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb34c06d300 0x7fb34c12bc80 crc :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.870+0000 7fb351230640 1 -- 172.21.15.60:0/3410508024 shutdown_connections 2023-08-24T10:08:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.870+0000 7fb351230640 1 --2- 172.21.15.60:0/3410508024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb34c06d300 0x7fb34c12bc80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.870+0000 7fb351230640 1 -- 172.21.15.60:0/3410508024 >> 172.21.15.60:0/3410508024 conn(0x7fb34c09e110 msgr2=0x7fb34c09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.870+0000 7fb351230640 1 -- 172.21.15.60:0/3410508024 shutdown_connections 2023-08-24T10:08:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.870+0000 7fb351230640 1 -- 172.21.15.60:0/3410508024 wait complete. 2023-08-24T10:08:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.870+0000 7fb351230640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.871+0000 7fb351230640 1 Processor -- start 2023-08-24T10:08:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.871+0000 7fb351230640 1 -- start start 2023-08-24T10:08:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.872+0000 7fb351230640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb34c06d300 0x7fb34c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.872+0000 7fb351230640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c089370 con 0x7fb34c06d300 2023-08-24T10:08:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.872+0000 7fb34ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb34c06d300 0x7fb34c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.872+0000 7fb34ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb34c06d300 0x7fb34c06e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37616/0 (socket says 172.21.15.60:37616) 2023-08-24T10:08:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.872+0000 7fb34ad9d640 1 -- 172.21.15.60:0/646059353 learned_addr learned my addr 172.21.15.60:0/646059353 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.872+0000 7fb34ad9d640 1 -- 172.21.15.60:0/646059353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c096f10 con 0x7fb34c06d300 2023-08-24T10:08:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.872+0000 7fb34ad9d640 1 --2- 172.21.15.60:0/646059353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb34c06d300 0x7fb34c06e9f0 crc :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.873+0000 7fb3337fe640 1 -- 172.21.15.60:0/646059353 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb33c018040 con 0x7fb34c06d300 2023-08-24T10:08:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.873+0000 7fb3337fe640 1 -- 172.21.15.60:0/646059353 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb33c00fda0 con 0x7fb34c06d300 2023-08-24T10:08:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.873+0000 7fb351230640 1 -- 172.21.15.60:0/646059353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34c089560 con 0x7fb34c06d300 2023-08-24T10:08:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.873+0000 7fb3337fe640 1 -- 172.21.15.60:0/646059353 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb33c013630 con 0x7fb34c06d300 2023-08-24T10:08:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.873+0000 7fb351230640 1 -- 172.21.15.60:0/646059353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34c088ff0 con 0x7fb34c06d300 2023-08-24T10:08:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.873+0000 7fb3337fe640 1 -- 172.21.15.60:0/646059353 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fb33c0137e0 con 0x7fb34c06d300 2023-08-24T10:08:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.874+0000 7fb3337fe640 1 --2- 172.21.15.60:0/646059353 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb338037130 0x7fb338039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:30.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.874+0000 7fb3337fe640 1 -- 172.21.15.60:0/646059353 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fb33c0495f0 con 0x7fb34c06d300 2023-08-24T10:08:30.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.874+0000 7fb351230640 1 -- 172.21.15.60:0/646059353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34c06e840 con 0x7fb34c06d300 2023-08-24T10:08:30.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.874+0000 7fb34a59c640 1 --2- 172.21.15.60:0/646059353 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb338037130 0x7fb338039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:30.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.874+0000 7fb34a59c640 1 --2- 172.21.15.60:0/646059353 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb338037130 0x7fb338039640 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:30.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:30.879+0000 7fb3337fe640 1 -- 172.21.15.60:0/646059353 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fb33c00d5d0 con 0x7fb34c06d300 2023-08-24T10:08:31.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:31.052+0000 7fb351230640 1 -- 172.21.15.60:0/646059353 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb34c06da20 con 0x7fb338037130 2023-08-24T10:08:31.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:31.054+0000 7fb3337fe640 1 -- 172.21.15.60:0/646059353 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7fb340001650 con 0x7fb338037130 2023-08-24T10:08:31.056 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:31.056 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:31.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:31.056+0000 7fb351230640 1 -- 172.21.15.60:0/646059353 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb338037130 msgr2=0x7fb338039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:31.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:31.056+0000 7fb351230640 1 --2- 172.21.15.60:0/646059353 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb338037130 0x7fb338039640 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:31.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:31.056+0000 7fb351230640 1 -- 172.21.15.60:0/646059353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb34c06d300 msgr2=0x7fb34c06e9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:31.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:31.056+0000 7fb351230640 1 --2- 172.21.15.60:0/646059353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb34c06d300 0x7fb34c06e9f0 crc :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:31.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:31.056+0000 7fb351230640 1 -- 172.21.15.60:0/646059353 shutdown_connections 2023-08-24T10:08:31.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:31.056+0000 7fb351230640 1 --2- 172.21.15.60:0/646059353 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb338037130 0x7fb338039640 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:31.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:31.056+0000 7fb351230640 1 --2- 172.21.15.60:0/646059353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb34c06d300 0x7fb34c06e9f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:31.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:31.056+0000 7fb351230640 1 -- 172.21.15.60:0/646059353 >> 172.21.15.60:0/646059353 conn(0x7fb34c09e110 msgr2=0x7fb34c09bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:31.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:31.056+0000 7fb351230640 1 -- 172.21.15.60:0/646059353 shutdown_connections 2023-08-24T10:08:31.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:31.056+0000 7fb351230640 1 -- 172.21.15.60:0/646059353 wait complete. 2023-08-24T10:08:31.069 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2023-08-24T10:08:29.535441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:34.073 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:34.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.193+0000 7f9323eac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:34.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.197+0000 7f9323eac640 1 Processor -- start 2023-08-24T10:08:34.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.197+0000 7f9323eac640 1 -- start start 2023-08-24T10:08:34.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.198+0000 7f9323eac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f931c131d40 0x7f931c134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:34.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.198+0000 7f9323eac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931c099440 con 0x7f931c06d300 2023-08-24T10:08:34.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.198+0000 7f9323eac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931c09a050 con 0x7f931c131d40 2023-08-24T10:08:34.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.198+0000 7f9321447640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f931c131d40 0x7f931c134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:34.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.198+0000 7f9321447640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f931c131d40 0x7f931c134160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37620/0 (socket says 172.21.15.60:37620) 2023-08-24T10:08:34.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.198+0000 7f9321447640 1 -- 172.21.15.60:0/4157681267 learned_addr learned my addr 172.21.15.60:0/4157681267 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:34.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.198+0000 7f9321c48640 1 --1- 172.21.15.60:0/4157681267 >> v1:172.21.15.60:6789/0 conn(0x7f931c06d300 0x7f931c12bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34444/0 (socket says 172.21.15.60:34444) 2023-08-24T10:08:34.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.198+0000 7f9321447640 1 -- 172.21.15.60:0/4157681267 >> v1:172.21.15.60:6789/0 conn(0x7f931c06d300 legacy=0x7f931c12bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:34.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.198+0000 7f9321447640 1 -- 172.21.15.60:0/4157681267 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c12b9d0 con 0x7f931c131d40 2023-08-24T10:08:34.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.198+0000 7f9321447640 1 --2- 172.21.15.60:0/4157681267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c131d40 0x7f931c134160 crc :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4858a0d045cc58 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:34.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.198+0000 7f9320c46640 1 -- 172.21.15.60:0/4157681267 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f930c009dc0 con 0x7f931c131d40 2023-08-24T10:08:34.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.198+0000 7f9320c46640 1 -- 172.21.15.60:0/4157681267 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f930c00e230 con 0x7f931c131d40 2023-08-24T10:08:34.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.198+0000 7f9320c46640 1 -- 172.21.15.60:0/4157681267 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f930c00e4d0 con 0x7f931c131d40 2023-08-24T10:08:34.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.199+0000 7f9323eac640 1 -- 172.21.15.60:0/4157681267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c131d40 msgr2=0x7f931c134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:34.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.199+0000 7f9323eac640 1 --2- 172.21.15.60:0/4157681267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c131d40 0x7f931c134160 crc :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:34.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.199+0000 7f9323eac640 1 -- 172.21.15.60:0/4157681267 shutdown_connections 2023-08-24T10:08:34.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.199+0000 7f9323eac640 1 --2- 172.21.15.60:0/4157681267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c131d40 0x7f931c134160 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:34.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.199+0000 7f9323eac640 1 -- 172.21.15.60:0/4157681267 >> 172.21.15.60:0/4157681267 conn(0x7f931c09e110 msgr2=0x7f931c09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:34.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.199+0000 7f9323eac640 1 -- 172.21.15.60:0/4157681267 shutdown_connections 2023-08-24T10:08:34.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.199+0000 7f9323eac640 1 -- 172.21.15.60:0/4157681267 wait complete. 2023-08-24T10:08:34.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.200+0000 7f9323eac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:34.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.200+0000 7f9323eac640 1 Processor -- start 2023-08-24T10:08:34.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.201+0000 7f9323eac640 1 -- start start 2023-08-24T10:08:34.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.201+0000 7f9323eac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c06d300 0x7f931c1356b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:34.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.201+0000 7f9323eac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c127be0 con 0x7f931c06d300 2023-08-24T10:08:34.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.201+0000 7f9321c48640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c06d300 0x7f931c1356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:34.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.201+0000 7f9321c48640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c06d300 0x7f931c1356b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37626/0 (socket says 172.21.15.60:37626) 2023-08-24T10:08:34.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.201+0000 7f9321c48640 1 -- 172.21.15.60:0/3864349406 learned_addr learned my addr 172.21.15.60:0/3864349406 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:34.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.202+0000 7f9321c48640 1 -- 172.21.15.60:0/3864349406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c071c80 con 0x7f931c06d300 2023-08-24T10:08:34.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.202+0000 7f9321c48640 1 --2- 172.21.15.60:0/3864349406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c06d300 0x7f931c1356b0 crc :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:34.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.202+0000 7f93127fc640 1 -- 172.21.15.60:0/3864349406 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f93180071e0 con 0x7f931c06d300 2023-08-24T10:08:34.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.202+0000 7f93127fc640 1 -- 172.21.15.60:0/3864349406 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f9318007340 con 0x7f931c06d300 2023-08-24T10:08:34.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.202+0000 7f9323eac640 1 -- 172.21.15.60:0/3864349406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931c128870 con 0x7f931c06d300 2023-08-24T10:08:34.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.202+0000 7f93127fc640 1 -- 172.21.15.60:0/3864349406 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f93180075e0 con 0x7f931c06d300 2023-08-24T10:08:34.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.202+0000 7f9323eac640 1 -- 172.21.15.60:0/3864349406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c134b10 con 0x7f931c06d300 2023-08-24T10:08:34.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.203+0000 7f93127fc640 1 -- 172.21.15.60:0/3864349406 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f93180057e0 con 0x7f931c06d300 2023-08-24T10:08:34.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.203+0000 7f9323eac640 1 -- 172.21.15.60:0/3864349406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931c071a80 con 0x7f931c06d300 2023-08-24T10:08:34.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.203+0000 7f93127fc640 1 --2- 172.21.15.60:0/3864349406 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9300037180 0x7f9300039690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:34.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.203+0000 7f93127fc640 1 -- 172.21.15.60:0/3864349406 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f9318046340 con 0x7f931c06d300 2023-08-24T10:08:34.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.204+0000 7f9321447640 1 --2- 172.21.15.60:0/3864349406 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9300037180 0x7f9300039690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:34.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.204+0000 7f9321447640 1 --2- 172.21.15.60:0/3864349406 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9300037180 0x7f9300039690 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:34.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.208+0000 7f93127fc640 1 -- 172.21.15.60:0/3864349406 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f931800d330 con 0x7f931c06d300 2023-08-24T10:08:34.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.379+0000 7f9323eac640 1 -- 172.21.15.60:0/3864349406 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f931c098880 con 0x7f9300037180 2023-08-24T10:08:34.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.380+0000 7f93127fc640 1 -- 172.21.15.60:0/3864349406 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f930c009d90 con 0x7f9300037180 2023-08-24T10:08:34.382 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:34.382 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:34.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.383+0000 7f9323eac640 1 -- 172.21.15.60:0/3864349406 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9300037180 msgr2=0x7f9300039690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:34.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.383+0000 7f9323eac640 1 --2- 172.21.15.60:0/3864349406 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9300037180 0x7f9300039690 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:34.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.383+0000 7f9323eac640 1 -- 172.21.15.60:0/3864349406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c06d300 msgr2=0x7f931c1356b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:34.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.383+0000 7f9323eac640 1 --2- 172.21.15.60:0/3864349406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c06d300 0x7f931c1356b0 crc :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:34.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.383+0000 7f9323eac640 1 -- 172.21.15.60:0/3864349406 shutdown_connections 2023-08-24T10:08:34.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.383+0000 7f9323eac640 1 --2- 172.21.15.60:0/3864349406 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9300037180 0x7f9300039690 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:34.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.383+0000 7f9323eac640 1 --2- 172.21.15.60:0/3864349406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f931c06d300 0x7f931c1356b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:34.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.383+0000 7f9323eac640 1 -- 172.21.15.60:0/3864349406 >> 172.21.15.60:0/3864349406 conn(0x7f931c09e110 msgr2=0x7f931c129720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:34.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.383+0000 7f9323eac640 1 -- 172.21.15.60:0/3864349406 shutdown_connections 2023-08-24T10:08:34.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.383+0000 7f9323eac640 1 -- 172.21.15.60:0/3864349406 wait complete. 2023-08-24T10:08:34.394 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2023-08-24T10:08:33.536259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:34.396 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:34.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.503+0000 7faeaa39e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:34.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.507+0000 7faeaa39e640 1 Processor -- start 2023-08-24T10:08:34.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.507+0000 7faeaa39e640 1 -- start start 2023-08-24T10:08:34.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.507+0000 7faeaa39e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faea4096860 0x7faea4096c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:34.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.507+0000 7faeaa39e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea40971b0 con 0x7faea40990b0 2023-08-24T10:08:34.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.507+0000 7faeaa39e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea406db70 con 0x7faea4096860 2023-08-24T10:08:34.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.507+0000 7faea3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faea4096860 0x7faea4096c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:34.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.508+0000 7faea3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faea4096860 0x7faea4096c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37634/0 (socket says 172.21.15.60:37634) 2023-08-24T10:08:34.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.508+0000 7faea3fff640 1 -- 172.21.15.60:0/835424336 learned_addr learned my addr 172.21.15.60:0/835424336 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:34.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.508+0000 7faea37fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7faea40990b0 0x7faea40994c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34458/0 (socket says 172.21.15.60:34458) 2023-08-24T10:08:34.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.508+0000 7faea3fff640 1 -- 172.21.15.60:0/835424336 >> v1:172.21.15.60:6789/0 conn(0x7faea40990b0 legacy=0x7faea40994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:34.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.508+0000 7faea3fff640 1 -- 172.21.15.60:0/835424336 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea406e320 con 0x7faea4096860 2023-08-24T10:08:34.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.508+0000 7faea3fff640 1 --2- 172.21.15.60:0/835424336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea4096860 0x7faea4096c70 crc :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e3aafc964f4993 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:34.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.508+0000 7faea2ffd640 1 -- 172.21.15.60:0/835424336 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fae94009dd0 con 0x7faea4096860 2023-08-24T10:08:34.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.508+0000 7faea2ffd640 1 -- 172.21.15.60:0/835424336 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fae9400e230 con 0x7faea4096860 2023-08-24T10:08:34.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.508+0000 7faea2ffd640 1 -- 172.21.15.60:0/835424336 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fae9400e4d0 con 0x7faea4096860 2023-08-24T10:08:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.509+0000 7faeaa39e640 1 -- 172.21.15.60:0/835424336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea4096860 msgr2=0x7faea4096c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.509+0000 7faeaa39e640 1 --2- 172.21.15.60:0/835424336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea4096860 0x7faea4096c70 crc :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.509+0000 7faeaa39e640 1 -- 172.21.15.60:0/835424336 shutdown_connections 2023-08-24T10:08:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.509+0000 7faeaa39e640 1 --2- 172.21.15.60:0/835424336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea4096860 0x7faea4096c70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:34.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.509+0000 7faeaa39e640 1 -- 172.21.15.60:0/835424336 >> 172.21.15.60:0/835424336 conn(0x7faea409e110 msgr2=0x7faea409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.509+0000 7faeaa39e640 1 -- 172.21.15.60:0/835424336 shutdown_connections 2023-08-24T10:08:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.509+0000 7faeaa39e640 1 -- 172.21.15.60:0/835424336 wait complete. 2023-08-24T10:08:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.509+0000 7faeaa39e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.510+0000 7faeaa39e640 1 Processor -- start 2023-08-24T10:08:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.510+0000 7faeaa39e640 1 -- start start 2023-08-24T10:08:34.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.511+0000 7faeaa39e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea4096860 0x7faea41c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.511+0000 7faeaa39e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea41c4e10 con 0x7faea4096860 2023-08-24T10:08:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.511+0000 7faea3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea4096860 0x7faea41c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.511+0000 7faea3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea4096860 0x7faea41c51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37650/0 (socket says 172.21.15.60:37650) 2023-08-24T10:08:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.511+0000 7faea3fff640 1 -- 172.21.15.60:0/1637908746 learned_addr learned my addr 172.21.15.60:0/1637908746 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.511+0000 7faea3fff640 1 -- 172.21.15.60:0/1637908746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea412bd60 con 0x7faea4096860 2023-08-24T10:08:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.511+0000 7faea3fff640 1 --2- 172.21.15.60:0/1637908746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea4096860 0x7faea41c51d0 crc :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.511+0000 7faea0ff9640 1 -- 172.21.15.60:0/1637908746 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fae94014410 con 0x7faea4096860 2023-08-24T10:08:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.511+0000 7faea0ff9640 1 -- 172.21.15.60:0/1637908746 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fae94014570 con 0x7faea4096860 2023-08-24T10:08:34.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.511+0000 7faeaa39e640 1 -- 172.21.15.60:0/1637908746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea412ae60 con 0x7faea4096860 2023-08-24T10:08:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.511+0000 7faea0ff9640 1 -- 172.21.15.60:0/1637908746 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fae94014810 con 0x7faea4096860 2023-08-24T10:08:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.511+0000 7faeaa39e640 1 -- 172.21.15.60:0/1637908746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea40989e0 con 0x7faea4096860 2023-08-24T10:08:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.512+0000 7faea0ff9640 1 -- 172.21.15.60:0/1637908746 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fae940149c0 con 0x7faea4096860 2023-08-24T10:08:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.512+0000 7faeaa39e640 1 -- 172.21.15.60:0/1637908746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea412a400 con 0x7faea4096860 2023-08-24T10:08:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.512+0000 7faea0ff9640 1 --2- 172.21.15.60:0/1637908746 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fae90037130 0x7fae90039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.512+0000 7faea0ff9640 1 -- 172.21.15.60:0/1637908746 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fae94019040 con 0x7faea4096860 2023-08-24T10:08:34.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.513+0000 7faea37fe640 1 --2- 172.21.15.60:0/1637908746 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fae90037130 0x7fae90039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:34.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.513+0000 7faea37fe640 1 --2- 172.21.15.60:0/1637908746 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fae90037130 0x7fae90039640 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:34.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.517+0000 7faea0ff9640 1 -- 172.21.15.60:0/1637908746 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fae9400f8f0 con 0x7faea4096860 2023-08-24T10:08:34.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.691+0000 7faeaa39e640 1 -- 172.21.15.60:0/1637908746 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faea412bb00 con 0x7fae90037130 2023-08-24T10:08:34.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.692+0000 7faea0ff9640 1 -- 172.21.15.60:0/1637908746 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7fae980070b0 con 0x7fae90037130 2023-08-24T10:08:34.694 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:34.695 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:34.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.694+0000 7faeaa39e640 1 -- 172.21.15.60:0/1637908746 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fae90037130 msgr2=0x7fae90039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:34.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.694+0000 7faeaa39e640 1 --2- 172.21.15.60:0/1637908746 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fae90037130 0x7fae90039640 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:34.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.694+0000 7faeaa39e640 1 -- 172.21.15.60:0/1637908746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea4096860 msgr2=0x7faea41c51d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:34.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.694+0000 7faeaa39e640 1 --2- 172.21.15.60:0/1637908746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea4096860 0x7faea41c51d0 crc :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:34.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.694+0000 7faeaa39e640 1 -- 172.21.15.60:0/1637908746 shutdown_connections 2023-08-24T10:08:34.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.694+0000 7faeaa39e640 1 --2- 172.21.15.60:0/1637908746 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fae90037130 0x7fae90039640 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:34.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.694+0000 7faeaa39e640 1 --2- 172.21.15.60:0/1637908746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faea4096860 0x7faea41c51d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:34.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.694+0000 7faeaa39e640 1 -- 172.21.15.60:0/1637908746 >> 172.21.15.60:0/1637908746 conn(0x7faea409e110 msgr2=0x7faea409bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:34.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.694+0000 7faeaa39e640 1 -- 172.21.15.60:0/1637908746 shutdown_connections 2023-08-24T10:08:34.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.694+0000 7faeaa39e640 1 -- 172.21.15.60:0/1637908746 wait complete. 2023-08-24T10:08:34.706 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2023-08-24T10:08:33.536259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:34.707 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:08:34.708 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:34.708 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:08:34.709 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:34.709 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:08:34.709 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:34.709 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:08:34.710 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:34.710 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:08:34.711 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:34.711 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:08:34.711 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:34.711 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:08:34.712 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:34.712 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:08:34.712 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:34.713 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:08:34.713 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:34.713 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:08:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.820+0000 7f12016ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.823+0000 7f12016ec640 1 Processor -- start 2023-08-24T10:08:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.823+0000 7f12016ec640 1 -- start start 2023-08-24T10:08:34.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.824+0000 7f12016ec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f11fc12f610 0x7f11fc12fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:34.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.824+0000 7f12016ec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11fc09d750 con 0x7f11fc1326f0 2023-08-24T10:08:34.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.824+0000 7f12016ec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc09d980 con 0x7f11fc12f610 2023-08-24T10:08:34.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.824+0000 7f11faffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f11fc12f610 0x7f11fc12fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:34.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.824+0000 7f11faffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f11fc12f610 0x7f11fc12fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37652/0 (socket says 172.21.15.60:37652) 2023-08-24T10:08:34.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.824+0000 7f11faffd640 1 -- 172.21.15.60:0/3364622599 learned_addr learned my addr 172.21.15.60:0/3364622599 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:34.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.824+0000 7f11fa7fc640 1 --1- 172.21.15.60:0/3364622599 >> v1:172.21.15.60:6789/0 conn(0x7f11fc1326f0 0x7f11fc132b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34460/0 (socket says 172.21.15.60:34460) 2023-08-24T10:08:34.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.825+0000 7f11faffd640 1 -- 172.21.15.60:0/3364622599 >> v1:172.21.15.60:6789/0 conn(0x7f11fc1326f0 legacy=0x7f11fc132b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:34.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.825+0000 7f11faffd640 1 -- 172.21.15.60:0/3364622599 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc09bde0 con 0x7f11fc12f610 2023-08-24T10:08:34.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.825+0000 7f11faffd640 1 --2- 172.21.15.60:0/3364622599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11fc12f610 0x7f11fc12fa20 crc :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb22fe02001b3fb9 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:34.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.825+0000 7f11f9ffb640 1 -- 172.21.15.60:0/3364622599 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f11ec00e040 con 0x7f11fc12f610 2023-08-24T10:08:34.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.825+0000 7f11f9ffb640 1 -- 172.21.15.60:0/3364622599 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f11ec0091e0 con 0x7f11fc12f610 2023-08-24T10:08:34.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.825+0000 7f11f9ffb640 1 -- 172.21.15.60:0/3364622599 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f11ec007730 con 0x7f11fc12f610 2023-08-24T10:08:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.825+0000 7f12016ec640 1 -- 172.21.15.60:0/3364622599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11fc12f610 msgr2=0x7f11fc12fa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.825+0000 7f12016ec640 1 --2- 172.21.15.60:0/3364622599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11fc12f610 0x7f11fc12fa20 crc :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.825+0000 7f12016ec640 1 -- 172.21.15.60:0/3364622599 shutdown_connections 2023-08-24T10:08:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.825+0000 7f12016ec640 1 --2- 172.21.15.60:0/3364622599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11fc12f610 0x7f11fc12fa20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.826+0000 7f12016ec640 1 -- 172.21.15.60:0/3364622599 >> 172.21.15.60:0/3364622599 conn(0x7f11fc0a2500 msgr2=0x7f11fc0a2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.826+0000 7f12016ec640 1 -- 172.21.15.60:0/3364622599 shutdown_connections 2023-08-24T10:08:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.826+0000 7f12016ec640 1 -- 172.21.15.60:0/3364622599 wait complete. 2023-08-24T10:08:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.826+0000 7f12016ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.827+0000 7f12016ec640 1 Processor -- start 2023-08-24T10:08:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.827+0000 7f12016ec640 1 -- start start 2023-08-24T10:08:34.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.827+0000 7f12016ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11fc12f610 0x7f11fc09fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:34.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.827+0000 7f12016ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc12cc60 con 0x7f11fc12f610 2023-08-24T10:08:34.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.828+0000 7f11faffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11fc12f610 0x7f11fc09fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:34.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.828+0000 7f11faffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11fc12f610 0x7f11fc09fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37662/0 (socket says 172.21.15.60:37662) 2023-08-24T10:08:34.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.828+0000 7f11faffd640 1 -- 172.21.15.60:0/1439730040 learned_addr learned my addr 172.21.15.60:0/1439730040 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:34.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.828+0000 7f11faffd640 1 -- 172.21.15.60:0/1439730040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc06a1f0 con 0x7f11fc12f610 2023-08-24T10:08:34.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.828+0000 7f11faffd640 1 --2- 172.21.15.60:0/1439730040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11fc12f610 0x7f11fc09fca0 crc :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:34.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.828+0000 7f11f37fe640 1 -- 172.21.15.60:0/1439730040 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f11ec00e040 con 0x7f11fc12f610 2023-08-24T10:08:34.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.828+0000 7f11f37fe640 1 -- 172.21.15.60:0/1439730040 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f11ec0094d0 con 0x7f11fc12f610 2023-08-24T10:08:34.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.828+0000 7f12016ec640 1 -- 172.21.15.60:0/1439730040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc09f7e0 con 0x7f11fc12f610 2023-08-24T10:08:34.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.828+0000 7f11f37fe640 1 -- 172.21.15.60:0/1439730040 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f11ec010990 con 0x7f11fc12f610 2023-08-24T10:08:34.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.828+0000 7f12016ec640 1 -- 172.21.15.60:0/1439730040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc09f4d0 con 0x7f11fc12f610 2023-08-24T10:08:34.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.829+0000 7f11f37fe640 1 -- 172.21.15.60:0/1439730040 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f11ec010b40 con 0x7f11fc12f610 2023-08-24T10:08:34.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.829+0000 7f11f37fe640 1 --2- 172.21.15.60:0/1439730040 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f11dc037130 0x7f11dc039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:34.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.829+0000 7f12016ec640 1 -- 172.21.15.60:0/1439730040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11fc061800 con 0x7f11fc12f610 2023-08-24T10:08:34.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.829+0000 7f11f37fe640 1 -- 172.21.15.60:0/1439730040 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f11ec047920 con 0x7f11fc12f610 2023-08-24T10:08:34.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.829+0000 7f11fa7fc640 1 --2- 172.21.15.60:0/1439730040 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f11dc037130 0x7f11dc039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:34.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.830+0000 7f11fa7fc640 1 --2- 172.21.15.60:0/1439730040 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f11dc037130 0x7f11dc039640 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:34.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:34.834+0000 7f11f37fe640 1 -- 172.21.15.60:0/1439730040 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f11ec015970 con 0x7f11fc12f610 2023-08-24T10:08:35.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.077+0000 7f12016ec640 1 -- 172.21.15.60:0/1439730040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f11fc06a1f0 con 0x7f11fc12f610 2023-08-24T10:08:35.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.077+0000 7f11f37fe640 1 -- 172.21.15.60:0/1439730040 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f11fc06a1f0 con 0x7f11fc12f610 2023-08-24T10:08:35.079 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:35.079 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":225,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:08:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.079+0000 7f12016ec640 1 -- 172.21.15.60:0/1439730040 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f11dc037130 msgr2=0x7f11dc039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.079+0000 7f12016ec640 1 --2- 172.21.15.60:0/1439730040 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f11dc037130 0x7f11dc039640 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.079+0000 7f12016ec640 1 -- 172.21.15.60:0/1439730040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11fc12f610 msgr2=0x7f11fc09fca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.079+0000 7f12016ec640 1 --2- 172.21.15.60:0/1439730040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11fc12f610 0x7f11fc09fca0 crc :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.079+0000 7f12016ec640 1 -- 172.21.15.60:0/1439730040 shutdown_connections 2023-08-24T10:08:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.079+0000 7f12016ec640 1 --2- 172.21.15.60:0/1439730040 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f11dc037130 0x7f11dc039640 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.079+0000 7f12016ec640 1 --2- 172.21.15.60:0/1439730040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11fc12f610 0x7f11fc09fca0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.079+0000 7f12016ec640 1 -- 172.21.15.60:0/1439730040 >> 172.21.15.60:0/1439730040 conn(0x7f11fc0a2500 msgr2=0x7f11fc1ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.079+0000 7f12016ec640 1 -- 172.21.15.60:0/1439730040 shutdown_connections 2023-08-24T10:08:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.079+0000 7f12016ec640 1 -- 172.21.15.60:0/1439730040 wait complete. 2023-08-24T10:08:35.091 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:08:35.092 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:35.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.198+0000 7f38d3955640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:35.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.202+0000 7f38d3955640 1 Processor -- start 2023-08-24T10:08:35.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.202+0000 7f38d3955640 1 -- start start 2023-08-24T10:08:35.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.203+0000 7f38d3955640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38cc070110 0x7f38cc1356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:35.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.203+0000 7f38d3955640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38cc096f10 con 0x7f38cc071e40 2023-08-24T10:08:35.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.203+0000 7f38d3955640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38cc0977f0 con 0x7f38cc070110 2023-08-24T10:08:35.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.203+0000 7f38d0ef0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38cc070110 0x7f38cc1356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:35.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.203+0000 7f38d0ef0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38cc070110 0x7f38cc1356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37670/0 (socket says 172.21.15.60:37670) 2023-08-24T10:08:35.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.203+0000 7f38d0ef0640 1 -- 172.21.15.60:0/849523605 learned_addr learned my addr 172.21.15.60:0/849523605 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:35.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.203+0000 7f38d16f1640 1 --1- 172.21.15.60:0/849523605 >> v1:172.21.15.60:6789/0 conn(0x7f38cc071e40 0x7f38cc096800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34476/0 (socket says 172.21.15.60:34476) 2023-08-24T10:08:35.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.203+0000 7f38d0ef0640 1 -- 172.21.15.60:0/849523605 >> v1:172.21.15.60:6789/0 conn(0x7f38cc071e40 legacy=0x7f38cc096800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:35.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.203+0000 7f38d0ef0640 1 -- 172.21.15.60:0/849523605 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc09a280 con 0x7f38cc070110 2023-08-24T10:08:35.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.204+0000 7f38d0ef0640 1 --2- 172.21.15.60:0/849523605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38cc070110 0x7f38cc1356a0 crc :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2cf93b128d0179 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:35.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.204+0000 7f38cbfff640 1 -- 172.21.15.60:0/849523605 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f38c0009dd0 con 0x7f38cc070110 2023-08-24T10:08:35.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.204+0000 7f38cbfff640 1 -- 172.21.15.60:0/849523605 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f38c000e230 con 0x7f38cc070110 2023-08-24T10:08:35.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.204+0000 7f38cbfff640 1 -- 172.21.15.60:0/849523605 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f38c000e4d0 con 0x7f38cc070110 2023-08-24T10:08:35.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.204+0000 7f38d3955640 1 -- 172.21.15.60:0/849523605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38cc070110 msgr2=0x7f38cc1356a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:35.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.204+0000 7f38d3955640 1 --2- 172.21.15.60:0/849523605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38cc070110 0x7f38cc1356a0 crc :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:35.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.204+0000 7f38d3955640 1 -- 172.21.15.60:0/849523605 shutdown_connections 2023-08-24T10:08:35.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.204+0000 7f38d3955640 1 --2- 172.21.15.60:0/849523605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38cc070110 0x7f38cc1356a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:35.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.204+0000 7f38d3955640 1 -- 172.21.15.60:0/849523605 >> 172.21.15.60:0/849523605 conn(0x7f38cc09e110 msgr2=0x7f38cc09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:35.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.204+0000 7f38d3955640 1 -- 172.21.15.60:0/849523605 shutdown_connections 2023-08-24T10:08:35.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.204+0000 7f38d3955640 1 -- 172.21.15.60:0/849523605 wait complete. 2023-08-24T10:08:35.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.205+0000 7f38d3955640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:35.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.206+0000 7f38d3955640 1 Processor -- start 2023-08-24T10:08:35.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.206+0000 7f38d3955640 1 -- start start 2023-08-24T10:08:35.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.206+0000 7f38d3955640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38cc070110 0x7f38cc093460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:35.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.206+0000 7f38d3955640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc08ff10 con 0x7f38cc070110 2023-08-24T10:08:35.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.206+0000 7f38d16f1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38cc070110 0x7f38cc093460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:35.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.206+0000 7f38d16f1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38cc070110 0x7f38cc093460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37678/0 (socket says 172.21.15.60:37678) 2023-08-24T10:08:35.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.206+0000 7f38d16f1640 1 -- 172.21.15.60:0/3628896535 learned_addr learned my addr 172.21.15.60:0/3628896535 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:35.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.207+0000 7f38d16f1640 1 -- 172.21.15.60:0/3628896535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc090680 con 0x7f38cc070110 2023-08-24T10:08:35.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.207+0000 7f38d16f1640 1 --2- 172.21.15.60:0/3628896535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38cc070110 0x7f38cc093460 crc :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:35.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.207+0000 7f38c9ffb640 1 -- 172.21.15.60:0/3628896535 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f38c40071e0 con 0x7f38cc070110 2023-08-24T10:08:35.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.207+0000 7f38c9ffb640 1 -- 172.21.15.60:0/3628896535 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f38c4007340 con 0x7f38cc070110 2023-08-24T10:08:35.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.207+0000 7f38d3955640 1 -- 172.21.15.60:0/3628896535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38cc092e80 con 0x7f38cc070110 2023-08-24T10:08:35.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.207+0000 7f38c9ffb640 1 -- 172.21.15.60:0/3628896535 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f38c40075e0 con 0x7f38cc070110 2023-08-24T10:08:35.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.207+0000 7f38d3955640 1 -- 172.21.15.60:0/3628896535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38cc092b70 con 0x7f38cc070110 2023-08-24T10:08:35.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.208+0000 7f38c9ffb640 1 -- 172.21.15.60:0/3628896535 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f38c40057e0 con 0x7f38cc070110 2023-08-24T10:08:35.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.208+0000 7f38c9ffb640 1 --2- 172.21.15.60:0/3628896535 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f38b8037130 0x7f38b8039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:35.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.208+0000 7f38c9ffb640 1 -- 172.21.15.60:0/3628896535 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f38c4046330 con 0x7f38cc070110 2023-08-24T10:08:35.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.208+0000 7f38d3955640 1 -- 172.21.15.60:0/3628896535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38cc061800 con 0x7f38cc070110 2023-08-24T10:08:35.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.208+0000 7f38d0ef0640 1 --2- 172.21.15.60:0/3628896535 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f38b8037130 0x7f38b8039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:35.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.209+0000 7f38d0ef0640 1 --2- 172.21.15.60:0/3628896535 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f38b8037130 0x7f38b8039640 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:35.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.213+0000 7f38c9ffb640 1 -- 172.21.15.60:0/3628896535 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f38c400d330 con 0x7f38cc070110 2023-08-24T10:08:35.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.397+0000 7f38d3955640 1 -- 172.21.15.60:0/3628896535 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f38cc0932f0 con 0x7f38b8037130 2023-08-24T10:08:35.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.399+0000 7f38c9ffb640 1 -- 172.21.15.60:0/3628896535 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f38cc0932f0 con 0x7f38b8037130 2023-08-24T10:08:35.401 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:35.403 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:35.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.400+0000 7f38d3955640 1 -- 172.21.15.60:0/3628896535 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f38b8037130 msgr2=0x7f38b8039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:35.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.400+0000 7f38d3955640 1 --2- 172.21.15.60:0/3628896535 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f38b8037130 0x7f38b8039640 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:35.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.400+0000 7f38d3955640 1 -- 172.21.15.60:0/3628896535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38cc070110 msgr2=0x7f38cc093460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:35.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.400+0000 7f38d3955640 1 --2- 172.21.15.60:0/3628896535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38cc070110 0x7f38cc093460 crc :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:35.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.400+0000 7f38d3955640 1 -- 172.21.15.60:0/3628896535 shutdown_connections 2023-08-24T10:08:35.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.400+0000 7f38d3955640 1 --2- 172.21.15.60:0/3628896535 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f38b8037130 0x7f38b8039640 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:35.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.400+0000 7f38d3955640 1 --2- 172.21.15.60:0/3628896535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38cc070110 0x7f38cc093460 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:35.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.400+0000 7f38d3955640 1 -- 172.21.15.60:0/3628896535 >> 172.21.15.60:0/3628896535 conn(0x7f38cc09e110 msgr2=0x7f38cc098490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:35.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.401+0000 7f38d3955640 1 -- 172.21.15.60:0/3628896535 shutdown_connections 2023-08-24T10:08:35.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:35.401+0000 7f38d3955640 1 -- 172.21.15.60:0/3628896535 wait complete. 2023-08-24T10:08:35.413 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2023-08-24T10:08:33.536259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:35.601 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~208s 2023-08-24T10:08:35.602 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~208s 2023-08-24T10:08:35.602 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~208s 2023-08-24T10:08:35.602 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~208s 2023-08-24T10:08:38.416 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:38.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.536+0000 7f1694f5e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:38.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.540+0000 7f1694f5e640 1 Processor -- start 2023-08-24T10:08:38.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.540+0000 7f1694f5e640 1 -- start start 2023-08-24T10:08:38.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.541+0000 7f1694f5e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1690071e40 0x7f1690096800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:38.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.541+0000 7f1694f5e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16900977f0 con 0x7f169006f100 2023-08-24T10:08:38.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.541+0000 7f1694f5e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f169009a280 con 0x7f1690071e40 2023-08-24T10:08:38.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.541+0000 7f168e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1690071e40 0x7f1690096800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:38.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.541+0000 7f168e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1690071e40 0x7f1690096800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57538/0 (socket says 172.21.15.60:57538) 2023-08-24T10:08:38.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.541+0000 7f168e59c640 1 -- 172.21.15.60:0/1796720598 learned_addr learned my addr 172.21.15.60:0/1796720598 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:38.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.541+0000 7f168e59c640 1 -- 172.21.15.60:0/1796720598 >> v1:172.21.15.60:6789/0 conn(0x7f169006f100 legacy=0x7f169006f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:38.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.541+0000 7f168e59c640 1 -- 172.21.15.60:0/1796720598 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169009a6b0 con 0x7f1690071e40 2023-08-24T10:08:38.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.541+0000 7f168e59c640 1 --2- 172.21.15.60:0/1796720598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1690071e40 0x7f1690096800 crc :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9208642c81b21bb3 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:38.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.542+0000 7f168d59a640 1 -- 172.21.15.60:0/1796720598 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1680009dd0 con 0x7f1690071e40 2023-08-24T10:08:38.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.542+0000 7f168d59a640 1 -- 172.21.15.60:0/1796720598 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f168000e230 con 0x7f1690071e40 2023-08-24T10:08:38.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.542+0000 7f168d59a640 1 -- 172.21.15.60:0/1796720598 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f168000e4d0 con 0x7f1690071e40 2023-08-24T10:08:38.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.542+0000 7f1694f5e640 1 -- 172.21.15.60:0/1796720598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1690071e40 msgr2=0x7f1690096800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:38.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.542+0000 7f1694f5e640 1 --2- 172.21.15.60:0/1796720598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1690071e40 0x7f1690096800 crc :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:38.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.542+0000 7f1694f5e640 1 -- 172.21.15.60:0/1796720598 shutdown_connections 2023-08-24T10:08:38.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.542+0000 7f1694f5e640 1 --2- 172.21.15.60:0/1796720598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1690071e40 0x7f1690096800 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:38.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.542+0000 7f1694f5e640 1 -- 172.21.15.60:0/1796720598 >> 172.21.15.60:0/1796720598 conn(0x7f169009e110 msgr2=0x7f169009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:38.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.542+0000 7f1694f5e640 1 -- 172.21.15.60:0/1796720598 shutdown_connections 2023-08-24T10:08:38.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.542+0000 7f1694f5e640 1 -- 172.21.15.60:0/1796720598 wait complete. 2023-08-24T10:08:38.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.543+0000 7f1694f5e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:38.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.544+0000 7f1694f5e640 1 Processor -- start 2023-08-24T10:08:38.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.544+0000 7f1694f5e640 1 -- start start 2023-08-24T10:08:38.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.544+0000 7f1694f5e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f169006f100 0x7f169006cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.544+0000 7f1694f5e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16900930d0 con 0x7f169006f100 2023-08-24T10:08:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.545+0000 7f168e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f169006f100 0x7f169006cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.545+0000 7f168e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f169006f100 0x7f169006cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57554/0 (socket says 172.21.15.60:57554) 2023-08-24T10:08:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.545+0000 7f168e59c640 1 -- 172.21.15.60:0/170567773 learned_addr learned my addr 172.21.15.60:0/170567773 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.545+0000 7f168e59c640 1 -- 172.21.15.60:0/170567773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1690125f70 con 0x7f169006f100 2023-08-24T10:08:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.545+0000 7f168e59c640 1 --2- 172.21.15.60:0/170567773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f169006f100 0x7f169006cef0 crc :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.545+0000 7f167effd640 1 -- 172.21.15.60:0/170567773 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1680014540 con 0x7f169006f100 2023-08-24T10:08:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.545+0000 7f167effd640 1 -- 172.21.15.60:0/170567773 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f16800146a0 con 0x7f169006f100 2023-08-24T10:08:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.545+0000 7f167effd640 1 -- 172.21.15.60:0/170567773 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1680014940 con 0x7f169006f100 2023-08-24T10:08:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.545+0000 7f1694f5e640 1 -- 172.21.15.60:0/170567773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1690125340 con 0x7f169006f100 2023-08-24T10:08:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.545+0000 7f1694f5e640 1 -- 172.21.15.60:0/170567773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1690131040 con 0x7f169006f100 2023-08-24T10:08:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.546+0000 7f167effd640 1 -- 172.21.15.60:0/170567773 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f1680014af0 con 0x7f169006f100 2023-08-24T10:08:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.546+0000 7f1694f5e640 1 -- 172.21.15.60:0/170567773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1690125530 con 0x7f169006f100 2023-08-24T10:08:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.546+0000 7f167effd640 1 --2- 172.21.15.60:0/170567773 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1670037130 0x7f1670039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.546+0000 7f167effd640 1 -- 172.21.15.60:0/170567773 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f1680019040 con 0x7f169006f100 2023-08-24T10:08:38.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.547+0000 7f168dd9b640 1 --2- 172.21.15.60:0/170567773 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1670037130 0x7f1670039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:38.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.547+0000 7f168dd9b640 1 --2- 172.21.15.60:0/170567773 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1670037130 0x7f1670039640 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:38.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.551+0000 7f167effd640 1 -- 172.21.15.60:0/170567773 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f1680018190 con 0x7f169006f100 2023-08-24T10:08:38.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.726+0000 7f1694f5e640 1 -- 172.21.15.60:0/170567773 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1690131290 con 0x7f1670037130 2023-08-24T10:08:38.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.727+0000 7f167effd640 1 -- 172.21.15.60:0/170567773 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f1690131290 con 0x7f1670037130 2023-08-24T10:08:38.730 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:38.731 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:38.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.729+0000 7f1694f5e640 1 -- 172.21.15.60:0/170567773 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1670037130 msgr2=0x7f1670039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:38.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.729+0000 7f1694f5e640 1 --2- 172.21.15.60:0/170567773 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1670037130 0x7f1670039640 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:38.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.729+0000 7f1694f5e640 1 -- 172.21.15.60:0/170567773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f169006f100 msgr2=0x7f169006cef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:38.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.729+0000 7f1694f5e640 1 --2- 172.21.15.60:0/170567773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f169006f100 0x7f169006cef0 crc :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:38.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.729+0000 7f1694f5e640 1 -- 172.21.15.60:0/170567773 shutdown_connections 2023-08-24T10:08:38.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.729+0000 7f1694f5e640 1 --2- 172.21.15.60:0/170567773 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1670037130 0x7f1670039640 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:38.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.729+0000 7f1694f5e640 1 --2- 172.21.15.60:0/170567773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f169006f100 0x7f169006cef0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:38.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.729+0000 7f1694f5e640 1 -- 172.21.15.60:0/170567773 >> 172.21.15.60:0/170567773 conn(0x7f169009e110 msgr2=0x7f16900985b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:38.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.729+0000 7f1694f5e640 1 -- 172.21.15.60:0/170567773 shutdown_connections 2023-08-24T10:08:38.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.729+0000 7f1694f5e640 1 -- 172.21.15.60:0/170567773 wait complete. 2023-08-24T10:08:38.741 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2023-08-24T10:08:37.537135+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:38.743 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.850+0000 7ffa17e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.854+0000 7ffa17e6c640 1 Processor -- start 2023-08-24T10:08:38.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.854+0000 7ffa17e6c640 1 -- start start 2023-08-24T10:08:38.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.855+0000 7ffa17e6c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffa101336f0 0x7ffa10135b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:38.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.855+0000 7ffa17e6c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa1006f720 con 0x7ffa10070410 2023-08-24T10:08:38.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.855+0000 7ffa17e6c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa1006fb30 con 0x7ffa101336f0 2023-08-24T10:08:38.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.855+0000 7ffa15407640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffa101336f0 0x7ffa10135b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:38.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.855+0000 7ffa15407640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffa101336f0 0x7ffa10135b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57560/0 (socket says 172.21.15.60:57560) 2023-08-24T10:08:38.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.855+0000 7ffa15407640 1 -- 172.21.15.60:0/832746207 learned_addr learned my addr 172.21.15.60:0/832746207 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:38.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.856+0000 7ffa15407640 1 -- 172.21.15.60:0/832746207 >> v1:172.21.15.60:6789/0 conn(0x7ffa10070410 legacy=0x7ffa10098a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:38.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.856+0000 7ffa15407640 1 -- 172.21.15.60:0/832746207 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa10070820 con 0x7ffa101336f0 2023-08-24T10:08:38.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.856+0000 7ffa15407640 1 --2- 172.21.15.60:0/832746207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa101336f0 0x7ffa10135b10 crc :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e107358fbc93966 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:38.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.856+0000 7ffa14c06640 1 -- 172.21.15.60:0/832746207 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ffa04009dd0 con 0x7ffa101336f0 2023-08-24T10:08:38.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.856+0000 7ffa14c06640 1 -- 172.21.15.60:0/832746207 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ffa0400e230 con 0x7ffa101336f0 2023-08-24T10:08:38.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.856+0000 7ffa14c06640 1 -- 172.21.15.60:0/832746207 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ffa0400e4d0 con 0x7ffa101336f0 2023-08-24T10:08:38.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.857+0000 7ffa17e6c640 1 -- 172.21.15.60:0/832746207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa101336f0 msgr2=0x7ffa10135b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:38.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.857+0000 7ffa17e6c640 1 --2- 172.21.15.60:0/832746207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa101336f0 0x7ffa10135b10 crc :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:38.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.857+0000 7ffa17e6c640 1 -- 172.21.15.60:0/832746207 shutdown_connections 2023-08-24T10:08:38.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.857+0000 7ffa17e6c640 1 --2- 172.21.15.60:0/832746207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa101336f0 0x7ffa10135b10 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:38.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.857+0000 7ffa17e6c640 1 -- 172.21.15.60:0/832746207 >> 172.21.15.60:0/832746207 conn(0x7ffa1009efd0 msgr2=0x7ffa1009f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:38.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.857+0000 7ffa17e6c640 1 -- 172.21.15.60:0/832746207 shutdown_connections 2023-08-24T10:08:38.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.857+0000 7ffa17e6c640 1 -- 172.21.15.60:0/832746207 wait complete. 2023-08-24T10:08:38.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.857+0000 7ffa17e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:38.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.858+0000 7ffa17e6c640 1 Processor -- start 2023-08-24T10:08:38.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.858+0000 7ffa17e6c640 1 -- start start 2023-08-24T10:08:38.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.859+0000 7ffa17e6c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa10070410 0x7ffa101c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:38.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.859+0000 7ffa17e6c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa10119790 con 0x7ffa10070410 2023-08-24T10:08:38.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.859+0000 7ffa15c08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa10070410 0x7ffa101c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:38.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.859+0000 7ffa15c08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa10070410 0x7ffa101c57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57566/0 (socket says 172.21.15.60:57566) 2023-08-24T10:08:38.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.859+0000 7ffa15c08640 1 -- 172.21.15.60:0/3678310936 learned_addr learned my addr 172.21.15.60:0/3678310936 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:38.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.859+0000 7ffa15c08640 1 -- 172.21.15.60:0/3678310936 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1009be90 con 0x7ffa10070410 2023-08-24T10:08:38.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.859+0000 7ffa15c08640 1 --2- 172.21.15.60:0/3678310936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa10070410 0x7ffa101c57b0 crc :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:38.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.859+0000 7ffa0e7fc640 1 -- 172.21.15.60:0/3678310936 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ffa080051e0 con 0x7ffa10070410 2023-08-24T10:08:38.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.859+0000 7ffa0e7fc640 1 -- 172.21.15.60:0/3678310936 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ffa08005340 con 0x7ffa10070410 2023-08-24T10:08:38.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.859+0000 7ffa17e6c640 1 -- 172.21.15.60:0/3678310936 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1011a3a0 con 0x7ffa10070410 2023-08-24T10:08:38.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.859+0000 7ffa0e7fc640 1 -- 172.21.15.60:0/3678310936 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ffa0800c730 con 0x7ffa10070410 2023-08-24T10:08:38.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.859+0000 7ffa17e6c640 1 -- 172.21.15.60:0/3678310936 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa101c5410 con 0x7ffa10070410 2023-08-24T10:08:38.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.860+0000 7ffa0e7fc640 1 -- 172.21.15.60:0/3678310936 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7ffa0800c920 con 0x7ffa10070410 2023-08-24T10:08:38.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.860+0000 7ffa17e6c640 1 -- 172.21.15.60:0/3678310936 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa10119550 con 0x7ffa10070410 2023-08-24T10:08:38.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.860+0000 7ffa0e7fc640 1 --2- 172.21.15.60:0/3678310936 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff9fc037130 0x7ff9fc039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:38.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.860+0000 7ffa0e7fc640 1 -- 172.21.15.60:0/3678310936 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7ffa08046370 con 0x7ffa10070410 2023-08-24T10:08:38.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.861+0000 7ffa15407640 1 --2- 172.21.15.60:0/3678310936 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff9fc037130 0x7ff9fc039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:38.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.861+0000 7ffa15407640 1 --2- 172.21.15.60:0/3678310936 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff9fc037130 0x7ff9fc039640 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:38.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:38.865+0000 7ffa0e7fc640 1 -- 172.21.15.60:0/3678310936 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7ffa0800d330 con 0x7ffa10070410 2023-08-24T10:08:39.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.035+0000 7ffa17e6c640 1 -- 172.21.15.60:0/3678310936 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffa1012d2b0 con 0x7ff9fc037130 2023-08-24T10:08:39.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.037+0000 7ffa0e7fc640 1 -- 172.21.15.60:0/3678310936 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7ffa04009d20 con 0x7ff9fc037130 2023-08-24T10:08:39.039 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:39.039 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:39.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.038+0000 7ffa17e6c640 1 -- 172.21.15.60:0/3678310936 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff9fc037130 msgr2=0x7ff9fc039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:39.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.038+0000 7ffa17e6c640 1 --2- 172.21.15.60:0/3678310936 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff9fc037130 0x7ff9fc039640 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:39.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.038+0000 7ffa17e6c640 1 -- 172.21.15.60:0/3678310936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa10070410 msgr2=0x7ffa101c57b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:39.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.038+0000 7ffa17e6c640 1 --2- 172.21.15.60:0/3678310936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa10070410 0x7ffa101c57b0 crc :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:39.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.038+0000 7ffa17e6c640 1 -- 172.21.15.60:0/3678310936 shutdown_connections 2023-08-24T10:08:39.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.038+0000 7ffa17e6c640 1 --2- 172.21.15.60:0/3678310936 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff9fc037130 0x7ff9fc039640 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:39.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.038+0000 7ffa17e6c640 1 --2- 172.21.15.60:0/3678310936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa10070410 0x7ffa101c57b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:39.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.038+0000 7ffa17e6c640 1 -- 172.21.15.60:0/3678310936 >> 172.21.15.60:0/3678310936 conn(0x7ffa1009efd0 msgr2=0x7ffa101b8810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:39.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.038+0000 7ffa17e6c640 1 -- 172.21.15.60:0/3678310936 shutdown_connections 2023-08-24T10:08:39.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.038+0000 7ffa17e6c640 1 -- 172.21.15.60:0/3678310936 wait complete. 2023-08-24T10:08:39.051 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2023-08-24T10:08:37.537135+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:39.053 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:08:39.054 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:39.055 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:08:39.055 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:39.055 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:08:39.055 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:39.056 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:08:39.056 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:39.056 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:08:39.056 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:39.057 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:08:39.057 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:39.057 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:08:39.057 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:39.058 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:08:39.058 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:39.058 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:08:39.058 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:39.059 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:08:39.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.165+0000 7fc99726c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:39.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.168+0000 7fc99726c640 1 Processor -- start 2023-08-24T10:08:39.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.169+0000 7fc99726c640 1 -- start start 2023-08-24T10:08:39.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.169+0000 7fc99726c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc99012f610 0x7fc99012fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:39.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.169+0000 7fc99726c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99009d750 con 0x7fc9901326f0 2023-08-24T10:08:39.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.169+0000 7fc99726c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc99009d980 con 0x7fc99012f610 2023-08-24T10:08:39.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.169+0000 7fc995008640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc99012f610 0x7fc99012fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:39.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.169+0000 7fc995008640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc99012f610 0x7fc99012fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57568/0 (socket says 172.21.15.60:57568) 2023-08-24T10:08:39.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.169+0000 7fc995008640 1 -- 172.21.15.60:0/726952185 learned_addr learned my addr 172.21.15.60:0/726952185 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:39.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.169+0000 7fc994807640 1 --1- 172.21.15.60:0/726952185 >> v1:172.21.15.60:6789/0 conn(0x7fc9901326f0 0x7fc990132b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57168/0 (socket says 172.21.15.60:57168) 2023-08-24T10:08:39.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.170+0000 7fc995008640 1 -- 172.21.15.60:0/726952185 >> v1:172.21.15.60:6789/0 conn(0x7fc9901326f0 legacy=0x7fc990132b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:39.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.170+0000 7fc995008640 1 -- 172.21.15.60:0/726952185 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99009bde0 con 0x7fc99012f610 2023-08-24T10:08:39.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.170+0000 7fc995008640 1 --2- 172.21.15.60:0/726952185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc99012f610 0x7fc99012fa20 crc :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e2551924fac983 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:39.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.170+0000 7fc987fff640 1 -- 172.21.15.60:0/726952185 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc98c009dc0 con 0x7fc99012f610 2023-08-24T10:08:39.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.170+0000 7fc987fff640 1 -- 172.21.15.60:0/726952185 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc98c00e230 con 0x7fc99012f610 2023-08-24T10:08:39.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.170+0000 7fc987fff640 1 -- 172.21.15.60:0/726952185 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc98c00e4d0 con 0x7fc99012f610 2023-08-24T10:08:39.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.171+0000 7fc99726c640 1 -- 172.21.15.60:0/726952185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc99012f610 msgr2=0x7fc99012fa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:39.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.171+0000 7fc99726c640 1 --2- 172.21.15.60:0/726952185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc99012f610 0x7fc99012fa20 crc :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:39.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.171+0000 7fc99726c640 1 -- 172.21.15.60:0/726952185 shutdown_connections 2023-08-24T10:08:39.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.171+0000 7fc99726c640 1 --2- 172.21.15.60:0/726952185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc99012f610 0x7fc99012fa20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:39.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.171+0000 7fc99726c640 1 -- 172.21.15.60:0/726952185 >> 172.21.15.60:0/726952185 conn(0x7fc9900a2500 msgr2=0x7fc9900a2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:39.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.171+0000 7fc99726c640 1 -- 172.21.15.60:0/726952185 shutdown_connections 2023-08-24T10:08:39.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.171+0000 7fc99726c640 1 -- 172.21.15.60:0/726952185 wait complete. 2023-08-24T10:08:39.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.171+0000 7fc99726c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:39.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.172+0000 7fc99726c640 1 Processor -- start 2023-08-24T10:08:39.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.172+0000 7fc99726c640 1 -- start start 2023-08-24T10:08:39.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.173+0000 7fc99726c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc99012f610 0x7fc99009fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:39.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.173+0000 7fc99726c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99012cc60 con 0x7fc99012f610 2023-08-24T10:08:39.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.173+0000 7fc995008640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc99012f610 0x7fc99009fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:39.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.173+0000 7fc995008640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc99012f610 0x7fc99009fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57576/0 (socket says 172.21.15.60:57576) 2023-08-24T10:08:39.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.173+0000 7fc995008640 1 -- 172.21.15.60:0/2982498073 learned_addr learned my addr 172.21.15.60:0/2982498073 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:39.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.173+0000 7fc995008640 1 -- 172.21.15.60:0/2982498073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99006a1f0 con 0x7fc99012f610 2023-08-24T10:08:39.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.173+0000 7fc995008640 1 --2- 172.21.15.60:0/2982498073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc99012f610 0x7fc99009fca0 crc :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:39.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.173+0000 7fc985ffb640 1 -- 172.21.15.60:0/2982498073 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc98c014540 con 0x7fc99012f610 2023-08-24T10:08:39.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.173+0000 7fc985ffb640 1 -- 172.21.15.60:0/2982498073 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc98c0146a0 con 0x7fc99012f610 2023-08-24T10:08:39.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.173+0000 7fc985ffb640 1 -- 172.21.15.60:0/2982498073 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc98c014940 con 0x7fc99012f610 2023-08-24T10:08:39.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.173+0000 7fc99726c640 1 -- 172.21.15.60:0/2982498073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99009f7e0 con 0x7fc99012f610 2023-08-24T10:08:39.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.174+0000 7fc99726c640 1 -- 172.21.15.60:0/2982498073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99009f4d0 con 0x7fc99012f610 2023-08-24T10:08:39.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.174+0000 7fc985ffb640 1 -- 172.21.15.60:0/2982498073 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fc98c014af0 con 0x7fc99012f610 2023-08-24T10:08:39.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.174+0000 7fc985ffb640 1 --2- 172.21.15.60:0/2982498073 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc974037180 0x7fc974039690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:39.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.174+0000 7fc985ffb640 1 -- 172.21.15.60:0/2982498073 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fc98c019040 con 0x7fc99012f610 2023-08-24T10:08:39.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.174+0000 7fc99726c640 1 -- 172.21.15.60:0/2982498073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99006a1f0 con 0x7fc99012f610 2023-08-24T10:08:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.175+0000 7fc994807640 1 --2- 172.21.15.60:0/2982498073 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc974037180 0x7fc974039690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.175+0000 7fc994807640 1 --2- 172.21.15.60:0/2982498073 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc974037180 0x7fc974039690 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.179+0000 7fc985ffb640 1 -- 172.21.15.60:0/2982498073 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fc99006a1f0 con 0x7fc99012f610 2023-08-24T10:08:39.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.424+0000 7fc99726c640 1 -- 172.21.15.60:0/2982498073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc99006a1f0 con 0x7fc99012f610 2023-08-24T10:08:39.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.424+0000 7fc985ffb640 1 -- 172.21.15.60:0/2982498073 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7fc99006a1f0 con 0x7fc99012f610 2023-08-24T10:08:39.426 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:39.426 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":230,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:08:39.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.426+0000 7fc99726c640 1 -- 172.21.15.60:0/2982498073 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc974037180 msgr2=0x7fc974039690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:39.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.426+0000 7fc99726c640 1 --2- 172.21.15.60:0/2982498073 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc974037180 0x7fc974039690 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:39.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.426+0000 7fc99726c640 1 -- 172.21.15.60:0/2982498073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc99012f610 msgr2=0x7fc99009fca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:39.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.426+0000 7fc99726c640 1 --2- 172.21.15.60:0/2982498073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc99012f610 0x7fc99009fca0 crc :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:39.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.426+0000 7fc99726c640 1 -- 172.21.15.60:0/2982498073 shutdown_connections 2023-08-24T10:08:39.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.426+0000 7fc99726c640 1 --2- 172.21.15.60:0/2982498073 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc974037180 0x7fc974039690 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:39.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.426+0000 7fc99726c640 1 --2- 172.21.15.60:0/2982498073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc99012f610 0x7fc99009fca0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:39.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.426+0000 7fc99726c640 1 -- 172.21.15.60:0/2982498073 >> 172.21.15.60:0/2982498073 conn(0x7fc9900a2500 msgr2=0x7fc9901ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:39.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.426+0000 7fc99726c640 1 -- 172.21.15.60:0/2982498073 shutdown_connections 2023-08-24T10:08:39.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.426+0000 7fc99726c640 1 -- 172.21.15.60:0/2982498073 wait complete. 2023-08-24T10:08:39.443 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:08:39.443 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:39.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.550+0000 7fd649e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:39.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.554+0000 7fd649e6c640 1 Processor -- start 2023-08-24T10:08:39.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.554+0000 7fd649e6c640 1 -- start start 2023-08-24T10:08:39.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.555+0000 7fd649e6c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd644131d40 0x7fd644134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:39.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.555+0000 7fd649e6c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd644099440 con 0x7fd64406d300 2023-08-24T10:08:39.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.555+0000 7fd649e6c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64409a050 con 0x7fd644131d40 2023-08-24T10:08:39.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.555+0000 7fd642ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd644131d40 0x7fd644134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:39.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.555+0000 7fd642ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd644131d40 0x7fd644134160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57578/0 (socket says 172.21.15.60:57578) 2023-08-24T10:08:39.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.555+0000 7fd642ffd640 1 -- 172.21.15.60:0/4010737952 learned_addr learned my addr 172.21.15.60:0/4010737952 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:39.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.555+0000 7fd642ffd640 1 -- 172.21.15.60:0/4010737952 >> v1:172.21.15.60:6789/0 conn(0x7fd64406d300 legacy=0x7fd64412bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:39.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.555+0000 7fd642ffd640 1 -- 172.21.15.60:0/4010737952 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64412b9d0 con 0x7fd644131d40 2023-08-24T10:08:39.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.555+0000 7fd642ffd640 1 --2- 172.21.15.60:0/4010737952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd644131d40 0x7fd644134160 crc :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8ced48f42c5362 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:39.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.555+0000 7fd6427fc640 1 -- 172.21.15.60:0/4010737952 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd638009dc0 con 0x7fd644131d40 2023-08-24T10:08:39.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.555+0000 7fd6427fc640 1 -- 172.21.15.60:0/4010737952 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd63800e230 con 0x7fd644131d40 2023-08-24T10:08:39.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.555+0000 7fd6427fc640 1 -- 172.21.15.60:0/4010737952 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd63800e4d0 con 0x7fd644131d40 2023-08-24T10:08:39.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.556+0000 7fd649e6c640 1 -- 172.21.15.60:0/4010737952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd644131d40 msgr2=0x7fd644134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:39.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.556+0000 7fd649e6c640 1 --2- 172.21.15.60:0/4010737952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd644131d40 0x7fd644134160 crc :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:39.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.556+0000 7fd649e6c640 1 -- 172.21.15.60:0/4010737952 shutdown_connections 2023-08-24T10:08:39.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.556+0000 7fd649e6c640 1 --2- 172.21.15.60:0/4010737952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd644131d40 0x7fd644134160 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:39.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.556+0000 7fd649e6c640 1 -- 172.21.15.60:0/4010737952 >> 172.21.15.60:0/4010737952 conn(0x7fd64409e110 msgr2=0x7fd64409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:39.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.556+0000 7fd649e6c640 1 -- 172.21.15.60:0/4010737952 shutdown_connections 2023-08-24T10:08:39.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.556+0000 7fd649e6c640 1 -- 172.21.15.60:0/4010737952 wait complete. 2023-08-24T10:08:39.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.556+0000 7fd649e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:39.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.557+0000 7fd649e6c640 1 Processor -- start 2023-08-24T10:08:39.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.557+0000 7fd649e6c640 1 -- start start 2023-08-24T10:08:39.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.558+0000 7fd649e6c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64406d300 0x7fd644096490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:39.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.558+0000 7fd649e6c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd644126360 con 0x7fd64406d300 2023-08-24T10:08:39.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.558+0000 7fd6437fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64406d300 0x7fd644096490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:39.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.558+0000 7fd6437fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64406d300 0x7fd644096490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57592/0 (socket says 172.21.15.60:57592) 2023-08-24T10:08:39.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.558+0000 7fd6437fe640 1 -- 172.21.15.60:0/2944463272 learned_addr learned my addr 172.21.15.60:0/2944463272 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:39.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.558+0000 7fd6437fe640 1 -- 172.21.15.60:0/2944463272 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd644134be0 con 0x7fd64406d300 2023-08-24T10:08:39.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.559+0000 7fd6437fe640 1 --2- 172.21.15.60:0/2944463272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64406d300 0x7fd644096490 crc :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:39.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.559+0000 7fd62bfff640 1 -- 172.21.15.60:0/2944463272 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd63400c860 con 0x7fd64406d300 2023-08-24T10:08:39.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.559+0000 7fd62bfff640 1 -- 172.21.15.60:0/2944463272 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd63400c9c0 con 0x7fd64406d300 2023-08-24T10:08:39.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.559+0000 7fd649e6c640 1 -- 172.21.15.60:0/2944463272 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd644128fb0 con 0x7fd64406d300 2023-08-24T10:08:39.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.559+0000 7fd62bfff640 1 -- 172.21.15.60:0/2944463272 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd63400cc60 con 0x7fd64406d300 2023-08-24T10:08:39.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.559+0000 7fd649e6c640 1 -- 172.21.15.60:0/2944463272 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd644126ca0 con 0x7fd64406d300 2023-08-24T10:08:39.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.560+0000 7fd62bfff640 1 -- 172.21.15.60:0/2944463272 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fd634011310 con 0x7fd64406d300 2023-08-24T10:08:39.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.560+0000 7fd649e6c640 1 -- 172.21.15.60:0/2944463272 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd644061800 con 0x7fd64406d300 2023-08-24T10:08:39.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.560+0000 7fd62bfff640 1 --2- 172.21.15.60:0/2944463272 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd630037130 0x7fd630039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:39.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.560+0000 7fd62bfff640 1 -- 172.21.15.60:0/2944463272 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fd6340469e0 con 0x7fd64406d300 2023-08-24T10:08:39.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.560+0000 7fd642ffd640 1 --2- 172.21.15.60:0/2944463272 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd630037130 0x7fd630039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:39.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.561+0000 7fd642ffd640 1 --2- 172.21.15.60:0/2944463272 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd630037130 0x7fd630039640 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:39.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.565+0000 7fd62bfff640 1 -- 172.21.15.60:0/2944463272 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fd634011e60 con 0x7fd64406d300 2023-08-24T10:08:39.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.736+0000 7fd649e6c640 1 -- 172.21.15.60:0/2944463272 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6441272c0 con 0x7fd630037130 2023-08-24T10:08:39.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.737+0000 7fd62bfff640 1 -- 172.21.15.60:0/2944463272 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7fd6441272c0 con 0x7fd630037130 2023-08-24T10:08:39.738 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:39.739 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:39.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.738+0000 7fd649e6c640 1 -- 172.21.15.60:0/2944463272 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd630037130 msgr2=0x7fd630039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:39.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.738+0000 7fd649e6c640 1 --2- 172.21.15.60:0/2944463272 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd630037130 0x7fd630039640 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:39.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.739+0000 7fd649e6c640 1 -- 172.21.15.60:0/2944463272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64406d300 msgr2=0x7fd644096490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:39.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.739+0000 7fd649e6c640 1 --2- 172.21.15.60:0/2944463272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64406d300 0x7fd644096490 crc :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:39.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.739+0000 7fd649e6c640 1 -- 172.21.15.60:0/2944463272 shutdown_connections 2023-08-24T10:08:39.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.739+0000 7fd649e6c640 1 --2- 172.21.15.60:0/2944463272 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd630037130 0x7fd630039640 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:39.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.739+0000 7fd649e6c640 1 --2- 172.21.15.60:0/2944463272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64406d300 0x7fd644096490 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:39.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.739+0000 7fd649e6c640 1 -- 172.21.15.60:0/2944463272 >> 172.21.15.60:0/2944463272 conn(0x7fd64409e110 msgr2=0x7fd644129720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:39.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.739+0000 7fd649e6c640 1 -- 172.21.15.60:0/2944463272 shutdown_connections 2023-08-24T10:08:39.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:39.739+0000 7fd649e6c640 1 -- 172.21.15.60:0/2944463272 wait complete. 2023-08-24T10:08:39.750 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2023-08-24T10:08:39.537495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:40.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~213s 2023-08-24T10:08:40.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~213s 2023-08-24T10:08:40.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~213s 2023-08-24T10:08:40.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~213s 2023-08-24T10:08:42.752 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:42.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.873+0000 7fb2f38ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:42.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.876+0000 7fb2f38ac640 1 Processor -- start 2023-08-24T10:08:42.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.877+0000 7fb2f38ac640 1 -- start start 2023-08-24T10:08:42.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.877+0000 7fb2f38ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb2ec131d40 0x7fb2ec134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:42.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.877+0000 7fb2f38ac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2ec099440 con 0x7fb2ec06d300 2023-08-24T10:08:42.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.877+0000 7fb2f38ac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2ec09a050 con 0x7fb2ec131d40 2023-08-24T10:08:42.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.878+0000 7fb2f1648640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb2ec06d300 0x7fb2ec12bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57190/0 (socket says 172.21.15.60:57190) 2023-08-24T10:08:42.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.878+0000 7fb2f1648640 1 -- 172.21.15.60:0/1730590102 learned_addr learned my addr 172.21.15.60:0/1730590102 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:08:42.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.878+0000 7fb2f0e47640 1 --2- 172.21.15.60:0/1730590102 >> v2:172.21.15.60:3300/0 conn(0x7fb2ec131d40 0x7fb2ec134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:42.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.878+0000 7fb2f0e47640 1 -- 172.21.15.60:0/1730590102 >> v1:172.21.15.60:6789/0 conn(0x7fb2ec06d300 legacy=0x7fb2ec12bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:42.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.878+0000 7fb2f0e47640 1 -- 172.21.15.60:0/1730590102 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec12b9d0 con 0x7fb2ec131d40 2023-08-24T10:08:42.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.878+0000 7fb2f0e47640 1 --2- 172.21.15.60:0/1730590102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2ec131d40 0x7fb2ec134160 crc :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949a864c87665b96 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:42.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.878+0000 7fb2e3fff640 1 -- 172.21.15.60:0/1730590102 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb2dc009cd0 con 0x7fb2ec131d40 2023-08-24T10:08:42.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.878+0000 7fb2e3fff640 1 -- 172.21.15.60:0/1730590102 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb2dc009e30 con 0x7fb2ec131d40 2023-08-24T10:08:42.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.878+0000 7fb2e3fff640 1 -- 172.21.15.60:0/1730590102 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb2dc00e360 con 0x7fb2ec131d40 2023-08-24T10:08:42.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.879+0000 7fb2f38ac640 1 -- 172.21.15.60:0/1730590102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2ec131d40 msgr2=0x7fb2ec134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:42.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.879+0000 7fb2f38ac640 1 --2- 172.21.15.60:0/1730590102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2ec131d40 0x7fb2ec134160 crc :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:42.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.879+0000 7fb2f38ac640 1 -- 172.21.15.60:0/1730590102 shutdown_connections 2023-08-24T10:08:42.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.879+0000 7fb2f38ac640 1 --2- 172.21.15.60:0/1730590102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2ec131d40 0x7fb2ec134160 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:42.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.879+0000 7fb2f38ac640 1 -- 172.21.15.60:0/1730590102 >> 172.21.15.60:0/1730590102 conn(0x7fb2ec09e110 msgr2=0x7fb2ec09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:42.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.879+0000 7fb2f38ac640 1 -- 172.21.15.60:0/1730590102 shutdown_connections 2023-08-24T10:08:42.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.879+0000 7fb2f38ac640 1 -- 172.21.15.60:0/1730590102 wait complete. 2023-08-24T10:08:42.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.879+0000 7fb2f38ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:42.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.880+0000 7fb2f38ac640 1 Processor -- start 2023-08-24T10:08:42.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.880+0000 7fb2f38ac640 1 -- start start 2023-08-24T10:08:42.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.881+0000 7fb2f38ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2ec06d300 0x7fb2ec06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:42.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.881+0000 7fb2f38ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec06ff10 con 0x7fb2ec06d300 2023-08-24T10:08:42.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.881+0000 7fb2f1648640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2ec06d300 0x7fb2ec06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:42.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.881+0000 7fb2f1648640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2ec06d300 0x7fb2ec06dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57608/0 (socket says 172.21.15.60:57608) 2023-08-24T10:08:42.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.881+0000 7fb2f1648640 1 -- 172.21.15.60:0/1878929086 learned_addr learned my addr 172.21.15.60:0/1878929086 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:42.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.881+0000 7fb2f1648640 1 -- 172.21.15.60:0/1878929086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec071c10 con 0x7fb2ec06d300 2023-08-24T10:08:42.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.881+0000 7fb2f1648640 1 --2- 172.21.15.60:0/1878929086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2ec06d300 0x7fb2ec06dc90 crc :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:42.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.882+0000 7fb2e1ffb640 1 -- 172.21.15.60:0/1878929086 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb2e80059d0 con 0x7fb2ec06d300 2023-08-24T10:08:42.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.882+0000 7fb2e1ffb640 1 -- 172.21.15.60:0/1878929086 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb2e8005b30 con 0x7fb2ec06d300 2023-08-24T10:08:42.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.882+0000 7fb2f38ac640 1 -- 172.21.15.60:0/1878929086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ec070660 con 0x7fb2ec06d300 2023-08-24T10:08:42.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.882+0000 7fb2e1ffb640 1 -- 172.21.15.60:0/1878929086 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb2e8005dd0 con 0x7fb2ec06d300 2023-08-24T10:08:42.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.882+0000 7fb2f38ac640 1 -- 172.21.15.60:0/1878929086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ec12b040 con 0x7fb2ec06d300 2023-08-24T10:08:42.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.883+0000 7fb2e1ffb640 1 -- 172.21.15.60:0/1878929086 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fb2e80070b0 con 0x7fb2ec06d300 2023-08-24T10:08:42.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.883+0000 7fb2e1ffb640 1 --2- 172.21.15.60:0/1878929086 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb2d0037130 0x7fb2d0039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:42.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.883+0000 7fb2e1ffb640 1 -- 172.21.15.60:0/1878929086 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fb2e80462f0 con 0x7fb2ec06d300 2023-08-24T10:08:42.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.883+0000 7fb2f38ac640 1 -- 172.21.15.60:0/1878929086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ec061800 con 0x7fb2ec06d300 2023-08-24T10:08:42.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.883+0000 7fb2f0e47640 1 --2- 172.21.15.60:0/1878929086 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb2d0037130 0x7fb2d0039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:42.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.884+0000 7fb2f0e47640 1 --2- 172.21.15.60:0/1878929086 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb2d0037130 0x7fb2d0039640 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:42.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:42.888+0000 7fb2e1ffb640 1 -- 172.21.15.60:0/1878929086 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fb2e800d720 con 0x7fb2ec06d300 2023-08-24T10:08:43.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.060+0000 7fb2f38ac640 1 -- 172.21.15.60:0/1878929086 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb2ec1363a0 con 0x7fb2d0037130 2023-08-24T10:08:43.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.061+0000 7fb2e1ffb640 1 -- 172.21.15.60:0/1878929086 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7fb2dc009c20 con 0x7fb2d0037130 2023-08-24T10:08:43.063 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:43.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.063+0000 7fb2f38ac640 1 -- 172.21.15.60:0/1878929086 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb2d0037130 msgr2=0x7fb2d0039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:43.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.063+0000 7fb2f38ac640 1 --2- 172.21.15.60:0/1878929086 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb2d0037130 0x7fb2d0039640 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.063+0000 7fb2f38ac640 1 -- 172.21.15.60:0/1878929086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2ec06d300 msgr2=0x7fb2ec06dc90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.063+0000 7fb2f38ac640 1 --2- 172.21.15.60:0/1878929086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2ec06d300 0x7fb2ec06dc90 crc :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.063+0000 7fb2f38ac640 1 -- 172.21.15.60:0/1878929086 shutdown_connections 2023-08-24T10:08:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.063+0000 7fb2f38ac640 1 --2- 172.21.15.60:0/1878929086 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb2d0037130 0x7fb2d0039640 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.063+0000 7fb2f38ac640 1 --2- 172.21.15.60:0/1878929086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2ec06d300 0x7fb2ec06dc90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.063+0000 7fb2f38ac640 1 -- 172.21.15.60:0/1878929086 >> 172.21.15.60:0/1878929086 conn(0x7fb2ec09e110 msgr2=0x7fb2ec097710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.063+0000 7fb2f38ac640 1 -- 172.21.15.60:0/1878929086 shutdown_connections 2023-08-24T10:08:43.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.063+0000 7fb2f38ac640 1 -- 172.21.15.60:0/1878929086 wait complete. 2023-08-24T10:08:43.077 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2023-08-24T10:08:41.538046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:43.078 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:43.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.184+0000 7f593c79e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:43.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.188+0000 7f593c79e640 1 Processor -- start 2023-08-24T10:08:43.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.188+0000 7f593c79e640 1 -- start start 2023-08-24T10:08:43.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.189+0000 7f593c79e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5934132b50 0x7f5934132f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:43.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.189+0000 7f593c79e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5934132610 con 0x7f593412f110 2023-08-24T10:08:43.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.189+0000 7f593c79e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f593409c260 con 0x7f5934132b50 2023-08-24T10:08:43.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.189+0000 7f593a53a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5934132b50 0x7f5934132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:43.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.189+0000 7f593a53a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5934132b50 0x7f5934132f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57616/0 (socket says 172.21.15.60:57616) 2023-08-24T10:08:43.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.189+0000 7f593a53a640 1 -- 172.21.15.60:0/2896235219 learned_addr learned my addr 172.21.15.60:0/2896235219 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:43.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.190+0000 7f593a53a640 1 -- 172.21.15.60:0/2896235219 >> v1:172.21.15.60:6789/0 conn(0x7f593412f110 legacy=0x7f593412f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:43.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.190+0000 7f593a53a640 1 -- 172.21.15.60:0/2896235219 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5934132840 con 0x7f5934132b50 2023-08-24T10:08:43.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.190+0000 7f593a53a640 1 --2- 172.21.15.60:0/2896235219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5934132b50 0x7f5934132f60 crc :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e8751b8721446d server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:43.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.190+0000 7f5939538640 1 -- 172.21.15.60:0/2896235219 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5930009dd0 con 0x7f5934132b50 2023-08-24T10:08:43.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.190+0000 7f5939538640 1 -- 172.21.15.60:0/2896235219 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f593000e230 con 0x7f5934132b50 2023-08-24T10:08:43.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.190+0000 7f5939538640 1 -- 172.21.15.60:0/2896235219 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f593000e4d0 con 0x7f5934132b50 2023-08-24T10:08:43.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.190+0000 7f593c79e640 1 -- 172.21.15.60:0/2896235219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5934132b50 msgr2=0x7f5934132f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:43.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.190+0000 7f593c79e640 1 --2- 172.21.15.60:0/2896235219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5934132b50 0x7f5934132f60 crc :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:43.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.191+0000 7f593c79e640 1 -- 172.21.15.60:0/2896235219 shutdown_connections 2023-08-24T10:08:43.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.191+0000 7f593c79e640 1 --2- 172.21.15.60:0/2896235219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5934132b50 0x7f5934132f60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:43.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.191+0000 7f593c79e640 1 -- 172.21.15.60:0/2896235219 >> 172.21.15.60:0/2896235219 conn(0x7f59340a26a0 msgr2=0x7f59340a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:43.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.191+0000 7f593c79e640 1 -- 172.21.15.60:0/2896235219 shutdown_connections 2023-08-24T10:08:43.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.191+0000 7f593c79e640 1 -- 172.21.15.60:0/2896235219 wait complete. 2023-08-24T10:08:43.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.191+0000 7f593c79e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:43.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.192+0000 7f593c79e640 1 Processor -- start 2023-08-24T10:08:43.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.192+0000 7f593c79e640 1 -- start start 2023-08-24T10:08:43.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.193+0000 7f593c79e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593412f110 0x7f593409fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:43.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.193+0000 7f593c79e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593412cc80 con 0x7f593412f110 2023-08-24T10:08:43.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.193+0000 7f593a53a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593412f110 0x7f593409fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:43.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.193+0000 7f593a53a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593412f110 0x7f593409fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57622/0 (socket says 172.21.15.60:57622) 2023-08-24T10:08:43.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.193+0000 7f593a53a640 1 -- 172.21.15.60:0/486669423 learned_addr learned my addr 172.21.15.60:0/486669423 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:43.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.193+0000 7f593a53a640 1 -- 172.21.15.60:0/486669423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593409f400 con 0x7f593412f110 2023-08-24T10:08:43.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.193+0000 7f593a53a640 1 --2- 172.21.15.60:0/486669423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593412f110 0x7f593409fcd0 crc :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:43.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.193+0000 7f592affd640 1 -- 172.21.15.60:0/486669423 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5930014410 con 0x7f593412f110 2023-08-24T10:08:43.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.193+0000 7f592affd640 1 -- 172.21.15.60:0/486669423 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5930014570 con 0x7f593412f110 2023-08-24T10:08:43.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.193+0000 7f593c79e640 1 -- 172.21.15.60:0/486669423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59340a18e0 con 0x7f593412f110 2023-08-24T10:08:43.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.193+0000 7f592affd640 1 -- 172.21.15.60:0/486669423 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5930014810 con 0x7f593412f110 2023-08-24T10:08:43.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.193+0000 7f593c79e640 1 -- 172.21.15.60:0/486669423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f593406a1d0 con 0x7f593412f110 2023-08-24T10:08:43.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.194+0000 7f592affd640 1 -- 172.21.15.60:0/486669423 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f59300149c0 con 0x7f593412f110 2023-08-24T10:08:43.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.194+0000 7f593c79e640 1 -- 172.21.15.60:0/486669423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f593411c2b0 con 0x7f593412f110 2023-08-24T10:08:43.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.194+0000 7f592affd640 1 --2- 172.21.15.60:0/486669423 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5918037130 0x7f5918039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:43.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.194+0000 7f592affd640 1 -- 172.21.15.60:0/486669423 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f5930019040 con 0x7f593412f110 2023-08-24T10:08:43.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.194+0000 7f5939d39640 1 --2- 172.21.15.60:0/486669423 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5918037130 0x7f5918039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:43.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.195+0000 7f5939d39640 1 --2- 172.21.15.60:0/486669423 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5918037130 0x7f5918039640 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:43.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.199+0000 7f592affd640 1 -- 172.21.15.60:0/486669423 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f593000f8f0 con 0x7f593412f110 2023-08-24T10:08:43.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.376+0000 7f593c79e640 1 -- 172.21.15.60:0/486669423 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f59341334a0 con 0x7f5918037130 2023-08-24T10:08:43.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.377+0000 7f592affd640 1 -- 172.21.15.60:0/486669423 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f5924001650 con 0x7f5918037130 2023-08-24T10:08:43.380 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.379+0000 7f593c79e640 1 -- 172.21.15.60:0/486669423 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5918037130 msgr2=0x7f5918039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.379+0000 7f593c79e640 1 --2- 172.21.15.60:0/486669423 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5918037130 0x7f5918039640 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.379+0000 7f593c79e640 1 -- 172.21.15.60:0/486669423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593412f110 msgr2=0x7f593409fcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.379+0000 7f593c79e640 1 --2- 172.21.15.60:0/486669423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593412f110 0x7f593409fcd0 crc :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.379+0000 7f593c79e640 1 -- 172.21.15.60:0/486669423 shutdown_connections 2023-08-24T10:08:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.379+0000 7f593c79e640 1 --2- 172.21.15.60:0/486669423 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5918037130 0x7f5918039640 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.379+0000 7f593c79e640 1 --2- 172.21.15.60:0/486669423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593412f110 0x7f593409fcd0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.379+0000 7f593c79e640 1 -- 172.21.15.60:0/486669423 >> 172.21.15.60:0/486669423 conn(0x7f59340a26a0 msgr2=0x7f59341ba920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.379+0000 7f593c79e640 1 -- 172.21.15.60:0/486669423 shutdown_connections 2023-08-24T10:08:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.379+0000 7f593c79e640 1 -- 172.21.15.60:0/486669423 wait complete. 2023-08-24T10:08:43.390 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2023-08-24T10:08:41.538046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:43.391 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:08:43.392 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:43.392 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:08:43.392 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:43.393 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:08:43.393 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:43.393 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:08:43.394 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:43.394 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:08:43.394 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:43.395 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:08:43.395 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:43.395 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:08:43.396 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:43.396 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:08:43.396 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:43.397 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:08:43.397 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:43.397 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:08:43.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.504+0000 7f4d48a30640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:43.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.507+0000 7f4d48a30640 1 Processor -- start 2023-08-24T10:08:43.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.507+0000 7f4d48a30640 1 -- start start 2023-08-24T10:08:43.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.508+0000 7f4d48a30640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4d44132680 0x7f4d44134aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:43.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.508+0000 7f4d48a30640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4406dfc0 con 0x7f4d44070c30 2023-08-24T10:08:43.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.508+0000 7f4d48a30640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4406d9b0 con 0x7f4d44132680 2023-08-24T10:08:43.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.508+0000 7f4d41d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4d44132680 0x7f4d44134aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:43.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.508+0000 7f4d41d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4d44132680 0x7f4d44134aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57632/0 (socket says 172.21.15.60:57632) 2023-08-24T10:08:43.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.508+0000 7f4d41d9b640 1 -- 172.21.15.60:0/3158068791 learned_addr learned my addr 172.21.15.60:0/3158068791 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:43.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.508+0000 7f4d4259c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4d44070c30 0x7f4d44071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57218/0 (socket says 172.21.15.60:57218) 2023-08-24T10:08:43.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.508+0000 7f4d41d9b640 1 -- 172.21.15.60:0/3158068791 >> v1:172.21.15.60:6789/0 conn(0x7f4d44070c30 legacy=0x7f4d44071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:43.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.508+0000 7f4d41d9b640 1 -- 172.21.15.60:0/3158068791 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4406e6e0 con 0x7f4d44132680 2023-08-24T10:08:43.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.508+0000 7f4d41d9b640 1 --2- 172.21.15.60:0/3158068791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d44132680 0x7f4d44134aa0 crc :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3984e331a0ad00c1 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:43.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.509+0000 7f4d4159a640 1 -- 172.21.15.60:0/3158068791 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4d2c009dd0 con 0x7f4d44132680 2023-08-24T10:08:43.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.509+0000 7f4d4159a640 1 -- 172.21.15.60:0/3158068791 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4d2c00e230 con 0x7f4d44132680 2023-08-24T10:08:43.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.509+0000 7f4d4159a640 1 -- 172.21.15.60:0/3158068791 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4d2c00e4d0 con 0x7f4d44132680 2023-08-24T10:08:43.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.509+0000 7f4d48a30640 1 -- 172.21.15.60:0/3158068791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d44132680 msgr2=0x7f4d44134aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:43.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.509+0000 7f4d48a30640 1 --2- 172.21.15.60:0/3158068791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d44132680 0x7f4d44134aa0 crc :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.509+0000 7f4d48a30640 1 -- 172.21.15.60:0/3158068791 shutdown_connections 2023-08-24T10:08:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.509+0000 7f4d48a30640 1 --2- 172.21.15.60:0/3158068791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d44132680 0x7f4d44134aa0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.509+0000 7f4d48a30640 1 -- 172.21.15.60:0/3158068791 >> 172.21.15.60:0/3158068791 conn(0x7f4d4409e110 msgr2=0x7f4d4409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.509+0000 7f4d48a30640 1 -- 172.21.15.60:0/3158068791 shutdown_connections 2023-08-24T10:08:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.509+0000 7f4d48a30640 1 -- 172.21.15.60:0/3158068791 wait complete. 2023-08-24T10:08:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.510+0000 7f4d48a30640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:43.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.511+0000 7f4d48a30640 1 Processor -- start 2023-08-24T10:08:43.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.511+0000 7f4d48a30640 1 -- start start 2023-08-24T10:08:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.511+0000 7f4d48a30640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d44070c30 0x7f4d4412d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.511+0000 7f4d48a30640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4412c970 con 0x7f4d44070c30 2023-08-24T10:08:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.511+0000 7f4d4259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d44070c30 0x7f4d4412d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.511+0000 7f4d4259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d44070c30 0x7f4d4412d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57634/0 (socket says 172.21.15.60:57634) 2023-08-24T10:08:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.511+0000 7f4d4259c640 1 -- 172.21.15.60:0/3874065060 learned_addr learned my addr 172.21.15.60:0/3874065060 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.512+0000 7f4d4259c640 1 -- 172.21.15.60:0/3874065060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4412d550 con 0x7f4d44070c30 2023-08-24T10:08:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.512+0000 7f4d4259c640 1 --2- 172.21.15.60:0/3874065060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d44070c30 0x7f4d4412d890 crc :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.512+0000 7f4d3affd640 1 -- 172.21.15.60:0/3874065060 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4d34005c90 con 0x7f4d44070c30 2023-08-24T10:08:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.512+0000 7f4d3affd640 1 -- 172.21.15.60:0/3874065060 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4d34005df0 con 0x7f4d44070c30 2023-08-24T10:08:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.512+0000 7f4d48a30640 1 -- 172.21.15.60:0/3874065060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4412cba0 con 0x7f4d44070c30 2023-08-24T10:08:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.512+0000 7f4d3affd640 1 -- 172.21.15.60:0/3874065060 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4d3400cba0 con 0x7f4d44070c30 2023-08-24T10:08:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.512+0000 7f4d48a30640 1 -- 172.21.15.60:0/3874065060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4412dfb0 con 0x7f4d44070c30 2023-08-24T10:08:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.513+0000 7f4d3affd640 1 -- 172.21.15.60:0/3874065060 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f4d340070b0 con 0x7f4d44070c30 2023-08-24T10:08:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.513+0000 7f4d3affd640 1 --2- 172.21.15.60:0/3874065060 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4d24037130 0x7f4d24039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.513+0000 7f4d3affd640 1 -- 172.21.15.60:0/3874065060 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f4d340466b0 con 0x7f4d44070c30 2023-08-24T10:08:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.513+0000 7f4d48a30640 1 -- 172.21.15.60:0/3874065060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d44061800 con 0x7f4d44070c30 2023-08-24T10:08:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.513+0000 7f4d41d9b640 1 --2- 172.21.15.60:0/3874065060 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4d24037130 0x7f4d24039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.514+0000 7f4d41d9b640 1 --2- 172.21.15.60:0/3874065060 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4d24037130 0x7f4d24039640 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:43.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.518+0000 7f4d3affd640 1 -- 172.21.15.60:0/3874065060 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f4d3400d7a0 con 0x7f4d44070c30 2023-08-24T10:08:43.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.768+0000 7f4d48a30640 1 -- 172.21.15.60:0/3874065060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f4d440706e0 con 0x7f4d44070c30 2023-08-24T10:08:43.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.769+0000 7f4d3affd640 1 -- 172.21.15.60:0/3874065060 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f4d3400d150 con 0x7f4d44070c30 2023-08-24T10:08:43.771 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:43.771 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":234,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:08:43.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.770+0000 7f4d48a30640 1 -- 172.21.15.60:0/3874065060 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4d24037130 msgr2=0x7f4d24039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:43.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.771+0000 7f4d48a30640 1 --2- 172.21.15.60:0/3874065060 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4d24037130 0x7f4d24039640 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:43.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.771+0000 7f4d48a30640 1 -- 172.21.15.60:0/3874065060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d44070c30 msgr2=0x7f4d4412d890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:43.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.771+0000 7f4d48a30640 1 --2- 172.21.15.60:0/3874065060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d44070c30 0x7f4d4412d890 crc :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:43.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.771+0000 7f4d48a30640 1 -- 172.21.15.60:0/3874065060 shutdown_connections 2023-08-24T10:08:43.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.771+0000 7f4d48a30640 1 --2- 172.21.15.60:0/3874065060 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4d24037130 0x7f4d24039640 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:43.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.771+0000 7f4d48a30640 1 --2- 172.21.15.60:0/3874065060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d44070c30 0x7f4d4412d890 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:43.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.771+0000 7f4d48a30640 1 -- 172.21.15.60:0/3874065060 >> 172.21.15.60:0/3874065060 conn(0x7f4d4409e110 msgr2=0x7f4d441b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:43.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.771+0000 7f4d48a30640 1 -- 172.21.15.60:0/3874065060 shutdown_connections 2023-08-24T10:08:43.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.771+0000 7f4d48a30640 1 -- 172.21.15.60:0/3874065060 wait complete. 2023-08-24T10:08:43.783 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:08:43.784 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:43.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.892+0000 7f63844ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:43.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.896+0000 7f63844ec640 1 Processor -- start 2023-08-24T10:08:43.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.896+0000 7f63844ec640 1 -- start start 2023-08-24T10:08:43.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.897+0000 7f63844ec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f637c131d40 0x7f637c134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:43.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.897+0000 7f63844ec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f637c099440 con 0x7f637c06d300 2023-08-24T10:08:43.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.897+0000 7f63844ec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f637c09a050 con 0x7f637c131d40 2023-08-24T10:08:43.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.897+0000 7f6381a87640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f637c131d40 0x7f637c134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:43.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.897+0000 7f6381a87640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f637c131d40 0x7f637c134160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57648/0 (socket says 172.21.15.60:57648) 2023-08-24T10:08:43.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.897+0000 7f6381a87640 1 -- 172.21.15.60:0/266063124 learned_addr learned my addr 172.21.15.60:0/266063124 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:43.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.897+0000 7f6382288640 1 --1- 172.21.15.60:0/266063124 >> v1:172.21.15.60:6789/0 conn(0x7f637c06d300 0x7f637c12bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57222/0 (socket says 172.21.15.60:57222) 2023-08-24T10:08:43.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.897+0000 7f6381a87640 1 -- 172.21.15.60:0/266063124 >> v1:172.21.15.60:6789/0 conn(0x7f637c06d300 legacy=0x7f637c12bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:43.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.897+0000 7f6381a87640 1 -- 172.21.15.60:0/266063124 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c12b9d0 con 0x7f637c131d40 2023-08-24T10:08:43.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.897+0000 7f6381a87640 1 --2- 172.21.15.60:0/266063124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c131d40 0x7f637c134160 crc :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b896d7c48c2c693 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:43.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.897+0000 7f6381286640 1 -- 172.21.15.60:0/266063124 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f636c009dc0 con 0x7f637c131d40 2023-08-24T10:08:43.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.897+0000 7f6381286640 1 -- 172.21.15.60:0/266063124 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f636c00e230 con 0x7f637c131d40 2023-08-24T10:08:43.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.897+0000 7f6381286640 1 -- 172.21.15.60:0/266063124 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f636c00e4d0 con 0x7f637c131d40 2023-08-24T10:08:43.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.898+0000 7f63844ec640 1 -- 172.21.15.60:0/266063124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c131d40 msgr2=0x7f637c134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:43.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.898+0000 7f63844ec640 1 --2- 172.21.15.60:0/266063124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c131d40 0x7f637c134160 crc :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:43.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.898+0000 7f63844ec640 1 -- 172.21.15.60:0/266063124 shutdown_connections 2023-08-24T10:08:43.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.898+0000 7f63844ec640 1 --2- 172.21.15.60:0/266063124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c131d40 0x7f637c134160 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:43.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.898+0000 7f63844ec640 1 -- 172.21.15.60:0/266063124 >> 172.21.15.60:0/266063124 conn(0x7f637c09e110 msgr2=0x7f637c09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:43.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.898+0000 7f63844ec640 1 -- 172.21.15.60:0/266063124 shutdown_connections 2023-08-24T10:08:43.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.898+0000 7f63844ec640 1 -- 172.21.15.60:0/266063124 wait complete. 2023-08-24T10:08:43.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.898+0000 7f63844ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:43.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.899+0000 7f63844ec640 1 Processor -- start 2023-08-24T10:08:43.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.899+0000 7f63844ec640 1 -- start start 2023-08-24T10:08:43.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.900+0000 7f63844ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c06d300 0x7f637c1356b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:43.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.900+0000 7f63844ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c127be0 con 0x7f637c06d300 2023-08-24T10:08:43.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.900+0000 7f6382288640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c06d300 0x7f637c1356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:43.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.900+0000 7f6382288640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c06d300 0x7f637c1356b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57660/0 (socket says 172.21.15.60:57660) 2023-08-24T10:08:43.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.900+0000 7f6382288640 1 -- 172.21.15.60:0/1623949618 learned_addr learned my addr 172.21.15.60:0/1623949618 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:43.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.900+0000 7f6382288640 1 -- 172.21.15.60:0/1623949618 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c071c80 con 0x7f637c06d300 2023-08-24T10:08:43.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.900+0000 7f6382288640 1 --2- 172.21.15.60:0/1623949618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c06d300 0x7f637c1356b0 crc :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:43.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.901+0000 7f6372ffd640 1 -- 172.21.15.60:0/1623949618 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f63780071e0 con 0x7f637c06d300 2023-08-24T10:08:43.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.901+0000 7f6372ffd640 1 -- 172.21.15.60:0/1623949618 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f6378007340 con 0x7f637c06d300 2023-08-24T10:08:43.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.901+0000 7f63844ec640 1 -- 172.21.15.60:0/1623949618 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637c128870 con 0x7f637c06d300 2023-08-24T10:08:43.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.901+0000 7f6372ffd640 1 -- 172.21.15.60:0/1623949618 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f63780075e0 con 0x7f637c06d300 2023-08-24T10:08:43.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.901+0000 7f63844ec640 1 -- 172.21.15.60:0/1623949618 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637c134b10 con 0x7f637c06d300 2023-08-24T10:08:43.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.902+0000 7f6372ffd640 1 -- 172.21.15.60:0/1623949618 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f63780057e0 con 0x7f637c06d300 2023-08-24T10:08:43.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.902+0000 7f63844ec640 1 -- 172.21.15.60:0/1623949618 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f637c071a80 con 0x7f637c06d300 2023-08-24T10:08:43.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.902+0000 7f6372ffd640 1 --2- 172.21.15.60:0/1623949618 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6360037130 0x7f6360039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:43.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.902+0000 7f6372ffd640 1 -- 172.21.15.60:0/1623949618 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f63780462f0 con 0x7f637c06d300 2023-08-24T10:08:43.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.902+0000 7f6381a87640 1 --2- 172.21.15.60:0/1623949618 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6360037130 0x7f6360039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:43.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.903+0000 7f6381a87640 1 --2- 172.21.15.60:0/1623949618 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6360037130 0x7f6360039640 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:43.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:43.906+0000 7f6372ffd640 1 -- 172.21.15.60:0/1623949618 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f637800d330 con 0x7f637c06d300 2023-08-24T10:08:44.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:44.082+0000 7f63844ec640 1 -- 172.21.15.60:0/1623949618 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f637c098880 con 0x7f6360037130 2023-08-24T10:08:44.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:44.083+0000 7f6372ffd640 1 -- 172.21.15.60:0/1623949618 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f636c009d90 con 0x7f6360037130 2023-08-24T10:08:44.084 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:44.085 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:44.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:44.085+0000 7f63844ec640 1 -- 172.21.15.60:0/1623949618 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6360037130 msgr2=0x7f6360039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:44.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:44.085+0000 7f63844ec640 1 --2- 172.21.15.60:0/1623949618 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6360037130 0x7f6360039640 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:44.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:44.085+0000 7f63844ec640 1 -- 172.21.15.60:0/1623949618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c06d300 msgr2=0x7f637c1356b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:44.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:44.085+0000 7f63844ec640 1 --2- 172.21.15.60:0/1623949618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c06d300 0x7f637c1356b0 crc :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:44.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:44.085+0000 7f63844ec640 1 -- 172.21.15.60:0/1623949618 shutdown_connections 2023-08-24T10:08:44.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:44.085+0000 7f63844ec640 1 --2- 172.21.15.60:0/1623949618 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6360037130 0x7f6360039640 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:44.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:44.085+0000 7f63844ec640 1 --2- 172.21.15.60:0/1623949618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637c06d300 0x7f637c1356b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:44.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:44.085+0000 7f63844ec640 1 -- 172.21.15.60:0/1623949618 >> 172.21.15.60:0/1623949618 conn(0x7f637c09e110 msgr2=0x7f637c129720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:44.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:44.085+0000 7f63844ec640 1 -- 172.21.15.60:0/1623949618 shutdown_connections 2023-08-24T10:08:44.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:44.085+0000 7f63844ec640 1 -- 172.21.15.60:0/1623949618 wait complete. 2023-08-24T10:08:44.097 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2023-08-24T10:08:43.538349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:46.010 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~219s 2023-08-24T10:08:46.011 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~219s 2023-08-24T10:08:46.011 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~219s 2023-08-24T10:08:46.011 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~219s 2023-08-24T10:08:47.099 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:47.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.220+0000 7f9cda8ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:47.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.224+0000 7f9cda8ac640 1 Processor -- start 2023-08-24T10:08:47.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.224+0000 7f9cda8ac640 1 -- start start 2023-08-24T10:08:47.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.225+0000 7f9cda8ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9cd4071e40 0x7f9cd4096800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:47.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.225+0000 7f9cda8ac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd40977f0 con 0x7f9cd406f100 2023-08-24T10:08:47.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.225+0000 7f9cda8ac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd409a280 con 0x7f9cd4071e40 2023-08-24T10:08:47.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.225+0000 7f9cd3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9cd4071e40 0x7f9cd4096800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:47.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.225+0000 7f9cd3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9cd4071e40 0x7f9cd4096800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33400/0 (socket says 172.21.15.60:33400) 2023-08-24T10:08:47.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.225+0000 7f9cd3fff640 1 -- 172.21.15.60:0/2010523292 learned_addr learned my addr 172.21.15.60:0/2010523292 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:47.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.225+0000 7f9cd3fff640 1 -- 172.21.15.60:0/2010523292 >> v1:172.21.15.60:6789/0 conn(0x7f9cd406f100 legacy=0x7f9cd406f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:47.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.225+0000 7f9cd3fff640 1 -- 172.21.15.60:0/2010523292 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd409a6b0 con 0x7f9cd4071e40 2023-08-24T10:08:47.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.225+0000 7f9cd3fff640 1 --2- 172.21.15.60:0/2010523292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cd4071e40 0x7f9cd4096800 crc :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f4cbf85d23d9fa server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:47.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.225+0000 7f9cd2ffd640 1 -- 172.21.15.60:0/2010523292 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9cc4009dd0 con 0x7f9cd4071e40 2023-08-24T10:08:47.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.226+0000 7f9cd2ffd640 1 -- 172.21.15.60:0/2010523292 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f9cc400e230 con 0x7f9cd4071e40 2023-08-24T10:08:47.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.226+0000 7f9cd2ffd640 1 -- 172.21.15.60:0/2010523292 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9cc400e4d0 con 0x7f9cd4071e40 2023-08-24T10:08:47.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.226+0000 7f9cda8ac640 1 -- 172.21.15.60:0/2010523292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cd4071e40 msgr2=0x7f9cd4096800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:47.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.226+0000 7f9cda8ac640 1 --2- 172.21.15.60:0/2010523292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cd4071e40 0x7f9cd4096800 crc :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:47.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.226+0000 7f9cda8ac640 1 -- 172.21.15.60:0/2010523292 shutdown_connections 2023-08-24T10:08:47.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.226+0000 7f9cda8ac640 1 --2- 172.21.15.60:0/2010523292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cd4071e40 0x7f9cd4096800 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:47.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.226+0000 7f9cda8ac640 1 -- 172.21.15.60:0/2010523292 >> 172.21.15.60:0/2010523292 conn(0x7f9cd409e110 msgr2=0x7f9cd409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:47.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.226+0000 7f9cda8ac640 1 -- 172.21.15.60:0/2010523292 shutdown_connections 2023-08-24T10:08:47.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.226+0000 7f9cda8ac640 1 -- 172.21.15.60:0/2010523292 wait complete. 2023-08-24T10:08:47.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.226+0000 7f9cda8ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:47.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.227+0000 7f9cda8ac640 1 Processor -- start 2023-08-24T10:08:47.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.228+0000 7f9cda8ac640 1 -- start start 2023-08-24T10:08:47.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.228+0000 7f9cda8ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cd406f100 0x7f9cd4092200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:47.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.228+0000 7f9cda8ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd4093e70 con 0x7f9cd406f100 2023-08-24T10:08:47.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.228+0000 7f9cd3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cd406f100 0x7f9cd4092200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:47.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.228+0000 7f9cd3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cd406f100 0x7f9cd4092200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33404/0 (socket says 172.21.15.60:33404) 2023-08-24T10:08:47.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.228+0000 7f9cd3fff640 1 -- 172.21.15.60:0/3313656644 learned_addr learned my addr 172.21.15.60:0/3313656644 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:47.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.229+0000 7f9cd3fff640 1 -- 172.21.15.60:0/3313656644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd406ec80 con 0x7f9cd406f100 2023-08-24T10:08:47.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.229+0000 7f9cd3fff640 1 --2- 172.21.15.60:0/3313656644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cd406f100 0x7f9cd4092200 crc :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:47.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.229+0000 7f9cd0ff9640 1 -- 172.21.15.60:0/3313656644 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9cc4014540 con 0x7f9cd406f100 2023-08-24T10:08:47.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.229+0000 7f9cd0ff9640 1 -- 172.21.15.60:0/3313656644 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f9cc40146a0 con 0x7f9cd406f100 2023-08-24T10:08:47.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.229+0000 7f9cda8ac640 1 -- 172.21.15.60:0/3313656644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd4091ef0 con 0x7f9cd406f100 2023-08-24T10:08:47.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.229+0000 7f9cd0ff9640 1 -- 172.21.15.60:0/3313656644 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9cc4014940 con 0x7f9cd406f100 2023-08-24T10:08:47.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.229+0000 7f9cda8ac640 1 -- 172.21.15.60:0/3313656644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd4090300 con 0x7f9cd406f100 2023-08-24T10:08:47.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.230+0000 7f9cd0ff9640 1 -- 172.21.15.60:0/3313656644 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f9cc4014af0 con 0x7f9cd406f100 2023-08-24T10:08:47.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.230+0000 7f9cda8ac640 1 -- 172.21.15.60:0/3313656644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd4097a50 con 0x7f9cd406f100 2023-08-24T10:08:47.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.230+0000 7f9cd0ff9640 1 --2- 172.21.15.60:0/3313656644 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9cc0037130 0x7f9cc0039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:47.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.230+0000 7f9cd0ff9640 1 -- 172.21.15.60:0/3313656644 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f9cc4019040 con 0x7f9cd406f100 2023-08-24T10:08:47.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.231+0000 7f9cd37fe640 1 --2- 172.21.15.60:0/3313656644 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9cc0037130 0x7f9cc0039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:47.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.231+0000 7f9cd37fe640 1 --2- 172.21.15.60:0/3313656644 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9cc0037130 0x7f9cc0039640 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:47.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.235+0000 7f9cd0ff9640 1 -- 172.21.15.60:0/3313656644 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f9cc4018190 con 0x7f9cd406f100 2023-08-24T10:08:47.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.410+0000 7f9cda8ac640 1 -- 172.21.15.60:0/3313656644 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9cd406b170 con 0x7f9cc0037130 2023-08-24T10:08:47.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.412+0000 7f9cd0ff9640 1 -- 172.21.15.60:0/3313656644 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f9cc8001650 con 0x7f9cc0037130 2023-08-24T10:08:47.414 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:47.416 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:47.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.414+0000 7f9cda8ac640 1 -- 172.21.15.60:0/3313656644 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9cc0037130 msgr2=0x7f9cc0039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:47.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.414+0000 7f9cda8ac640 1 --2- 172.21.15.60:0/3313656644 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9cc0037130 0x7f9cc0039640 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:47.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.414+0000 7f9cda8ac640 1 -- 172.21.15.60:0/3313656644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cd406f100 msgr2=0x7f9cd4092200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:47.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.414+0000 7f9cda8ac640 1 --2- 172.21.15.60:0/3313656644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cd406f100 0x7f9cd4092200 crc :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:47.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.414+0000 7f9cda8ac640 1 -- 172.21.15.60:0/3313656644 shutdown_connections 2023-08-24T10:08:47.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.414+0000 7f9cda8ac640 1 --2- 172.21.15.60:0/3313656644 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9cc0037130 0x7f9cc0039640 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:47.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.414+0000 7f9cda8ac640 1 --2- 172.21.15.60:0/3313656644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cd406f100 0x7f9cd4092200 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:47.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.414+0000 7f9cda8ac640 1 -- 172.21.15.60:0/3313656644 >> 172.21.15.60:0/3313656644 conn(0x7f9cd409e110 msgr2=0x7f9cd4098540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:47.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.414+0000 7f9cda8ac640 1 -- 172.21.15.60:0/3313656644 shutdown_connections 2023-08-24T10:08:47.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.414+0000 7f9cda8ac640 1 -- 172.21.15.60:0/3313656644 wait complete. 2023-08-24T10:08:47.426 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2023-08-24T10:08:45.538671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:47.427 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:47.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.533+0000 7f974751e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:47.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.537+0000 7f974751e640 1 Processor -- start 2023-08-24T10:08:47.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.537+0000 7f974751e640 1 -- start start 2023-08-24T10:08:47.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.537+0000 7f974751e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9740071e40 0x7f9740096800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:47.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.538+0000 7f974751e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97400977f0 con 0x7f974006f100 2023-08-24T10:08:47.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.538+0000 7f974751e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f974009a280 con 0x7f9740071e40 2023-08-24T10:08:47.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.538+0000 7f97452ba640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9740071e40 0x7f9740096800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:47.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.538+0000 7f97452ba640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9740071e40 0x7f9740096800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33406/0 (socket says 172.21.15.60:33406) 2023-08-24T10:08:47.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.538+0000 7f97452ba640 1 -- 172.21.15.60:0/1946439598 learned_addr learned my addr 172.21.15.60:0/1946439598 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:47.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.538+0000 7f9744ab9640 1 --1- 172.21.15.60:0/1946439598 >> v1:172.21.15.60:6789/0 conn(0x7f974006f100 0x7f974006f510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57336/0 (socket says 172.21.15.60:57336) 2023-08-24T10:08:47.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.538+0000 7f97452ba640 1 -- 172.21.15.60:0/1946439598 >> v1:172.21.15.60:6789/0 conn(0x7f974006f100 legacy=0x7f974006f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:47.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.538+0000 7f97452ba640 1 -- 172.21.15.60:0/1946439598 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974009a6b0 con 0x7f9740071e40 2023-08-24T10:08:47.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.538+0000 7f97452ba640 1 --2- 172.21.15.60:0/1946439598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9740071e40 0x7f9740096800 crc :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a100df017c60ed46 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:47.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.539+0000 7f9737fff640 1 -- 172.21.15.60:0/1946439598 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f973c009dd0 con 0x7f9740071e40 2023-08-24T10:08:47.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.539+0000 7f9737fff640 1 -- 172.21.15.60:0/1946439598 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f973c00e230 con 0x7f9740071e40 2023-08-24T10:08:47.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.539+0000 7f9737fff640 1 -- 172.21.15.60:0/1946439598 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f973c00e4d0 con 0x7f9740071e40 2023-08-24T10:08:47.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.539+0000 7f974751e640 1 -- 172.21.15.60:0/1946439598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9740071e40 msgr2=0x7f9740096800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:47.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.539+0000 7f974751e640 1 --2- 172.21.15.60:0/1946439598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9740071e40 0x7f9740096800 crc :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:47.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.539+0000 7f974751e640 1 -- 172.21.15.60:0/1946439598 shutdown_connections 2023-08-24T10:08:47.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.539+0000 7f974751e640 1 --2- 172.21.15.60:0/1946439598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9740071e40 0x7f9740096800 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:47.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.539+0000 7f974751e640 1 -- 172.21.15.60:0/1946439598 >> 172.21.15.60:0/1946439598 conn(0x7f974009e110 msgr2=0x7f974009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:47.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.539+0000 7f974751e640 1 -- 172.21.15.60:0/1946439598 shutdown_connections 2023-08-24T10:08:47.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.539+0000 7f974751e640 1 -- 172.21.15.60:0/1946439598 wait complete. 2023-08-24T10:08:47.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.540+0000 7f974751e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:47.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.540+0000 7f974751e640 1 Processor -- start 2023-08-24T10:08:47.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.541+0000 7f974751e640 1 -- start start 2023-08-24T10:08:47.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.541+0000 7f974751e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f974006f100 0x7f97401c19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:47.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.541+0000 7f974751e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974008ffe0 con 0x7f974006f100 2023-08-24T10:08:47.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.541+0000 7f97452ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f974006f100 0x7f97401c19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:47.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.541+0000 7f97452ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f974006f100 0x7f97401c19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33420/0 (socket says 172.21.15.60:33420) 2023-08-24T10:08:47.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.541+0000 7f97452ba640 1 -- 172.21.15.60:0/528025685 learned_addr learned my addr 172.21.15.60:0/528025685 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:47.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.542+0000 7f97452ba640 1 -- 172.21.15.60:0/528025685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97401c14b0 con 0x7f974006f100 2023-08-24T10:08:47.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.542+0000 7f97452ba640 1 --2- 172.21.15.60:0/528025685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f974006f100 0x7f97401c19f0 crc :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:47.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.542+0000 7f9735ffb640 1 -- 172.21.15.60:0/528025685 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f973c014540 con 0x7f974006f100 2023-08-24T10:08:47.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.542+0000 7f9735ffb640 1 -- 172.21.15.60:0/528025685 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f973c0146a0 con 0x7f974006f100 2023-08-24T10:08:47.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.542+0000 7f9735ffb640 1 -- 172.21.15.60:0/528025685 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f973c014940 con 0x7f974006f100 2023-08-24T10:08:47.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.542+0000 7f974751e640 1 -- 172.21.15.60:0/528025685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9740090720 con 0x7f974006f100 2023-08-24T10:08:47.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.542+0000 7f974751e640 1 -- 172.21.15.60:0/528025685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9740092c10 con 0x7f974006f100 2023-08-24T10:08:47.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.543+0000 7f9735ffb640 1 -- 172.21.15.60:0/528025685 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f973c014af0 con 0x7f974006f100 2023-08-24T10:08:47.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.543+0000 7f9735ffb640 1 --2- 172.21.15.60:0/528025685 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9724037130 0x7f9724039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:47.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.543+0000 7f9735ffb640 1 -- 172.21.15.60:0/528025685 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f973c019040 con 0x7f974006f100 2023-08-24T10:08:47.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.543+0000 7f974751e640 1 -- 172.21.15.60:0/528025685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9740061800 con 0x7f974006f100 2023-08-24T10:08:47.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.544+0000 7f9744ab9640 1 --2- 172.21.15.60:0/528025685 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9724037130 0x7f9724039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:47.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.544+0000 7f9744ab9640 1 --2- 172.21.15.60:0/528025685 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9724037130 0x7f9724039640 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:47.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.548+0000 7f9735ffb640 1 -- 172.21.15.60:0/528025685 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f973c018190 con 0x7f974006f100 2023-08-24T10:08:47.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.719+0000 7f974751e640 1 -- 172.21.15.60:0/528025685 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9740097a80 con 0x7f9724037130 2023-08-24T10:08:47.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.721+0000 7f9735ffb640 1 -- 172.21.15.60:0/528025685 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f97300070b0 con 0x7f9724037130 2023-08-24T10:08:47.722 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:47.722 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:47.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.722+0000 7f974751e640 1 -- 172.21.15.60:0/528025685 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9724037130 msgr2=0x7f9724039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:47.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.722+0000 7f974751e640 1 --2- 172.21.15.60:0/528025685 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9724037130 0x7f9724039640 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:47.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.723+0000 7f974751e640 1 -- 172.21.15.60:0/528025685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f974006f100 msgr2=0x7f97401c19f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:47.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.723+0000 7f974751e640 1 --2- 172.21.15.60:0/528025685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f974006f100 0x7f97401c19f0 crc :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:47.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.723+0000 7f974751e640 1 -- 172.21.15.60:0/528025685 shutdown_connections 2023-08-24T10:08:47.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.723+0000 7f974751e640 1 --2- 172.21.15.60:0/528025685 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9724037130 0x7f9724039640 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:47.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.723+0000 7f974751e640 1 --2- 172.21.15.60:0/528025685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f974006f100 0x7f97401c19f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:47.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.723+0000 7f974751e640 1 -- 172.21.15.60:0/528025685 >> 172.21.15.60:0/528025685 conn(0x7f974009e110 msgr2=0x7f97400985b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:47.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.723+0000 7f974751e640 1 -- 172.21.15.60:0/528025685 shutdown_connections 2023-08-24T10:08:47.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.723+0000 7f974751e640 1 -- 172.21.15.60:0/528025685 wait complete. 2023-08-24T10:08:47.734 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2023-08-24T10:08:47.539020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:47.735 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:08:47.736 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:47.736 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:08:47.736 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:47.736 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:08:47.737 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:47.737 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:08:47.737 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:47.738 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:08:47.738 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:47.738 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:08:47.739 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:47.739 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:08:47.740 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:47.740 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:08:47.741 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:47.741 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:08:47.741 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:47.741 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:08:47.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.848+0000 7f445751e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:47.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.851+0000 7f445751e640 1 Processor -- start 2023-08-24T10:08:47.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.851+0000 7f445751e640 1 -- start start 2023-08-24T10:08:47.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.852+0000 7f445751e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f445012ece0 0x7f445012f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:47.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.852+0000 7f445751e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f445009a650 con 0x7f445012f630 2023-08-24T10:08:47.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.852+0000 7f445751e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f445006da00 con 0x7f445012ece0 2023-08-24T10:08:47.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.852+0000 7f44552ba640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f445012ece0 0x7f445012f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:47.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.852+0000 7f44552ba640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f445012ece0 0x7f445012f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33432/0 (socket says 172.21.15.60:33432) 2023-08-24T10:08:47.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.852+0000 7f44552ba640 1 -- 172.21.15.60:0/479073807 learned_addr learned my addr 172.21.15.60:0/479073807 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:47.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.852+0000 7f4454ab9640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f445012f630 0x7f4450097fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57346/0 (socket says 172.21.15.60:57346) 2023-08-24T10:08:47.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.852+0000 7f44552ba640 1 -- 172.21.15.60:0/479073807 >> v1:172.21.15.60:6789/0 conn(0x7f445012f630 legacy=0x7f4450097fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:47.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.852+0000 7f44552ba640 1 -- 172.21.15.60:0/479073807 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445012ea10 con 0x7f445012ece0 2023-08-24T10:08:47.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.852+0000 7f44552ba640 1 --2- 172.21.15.60:0/479073807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f445012ece0 0x7f445012f0f0 crc :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f63d7108a0415e server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:47.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.853+0000 7f444ffff640 1 -- 172.21.15.60:0/479073807 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f444800f040 con 0x7f445012ece0 2023-08-24T10:08:47.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.853+0000 7f444ffff640 1 -- 172.21.15.60:0/479073807 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f444800a1e0 con 0x7f445012ece0 2023-08-24T10:08:47.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.853+0000 7f444ffff640 1 -- 172.21.15.60:0/479073807 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4448008710 con 0x7f445012ece0 2023-08-24T10:08:47.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.853+0000 7f445751e640 1 -- 172.21.15.60:0/479073807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f445012ece0 msgr2=0x7f445012f0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:47.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.853+0000 7f445751e640 1 --2- 172.21.15.60:0/479073807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f445012ece0 0x7f445012f0f0 crc :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:47.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.853+0000 7f445751e640 1 -- 172.21.15.60:0/479073807 shutdown_connections 2023-08-24T10:08:47.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.853+0000 7f445751e640 1 --2- 172.21.15.60:0/479073807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f445012ece0 0x7f445012f0f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:47.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.853+0000 7f445751e640 1 -- 172.21.15.60:0/479073807 >> 172.21.15.60:0/479073807 conn(0x7f445009e110 msgr2=0x7f445009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:47.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.853+0000 7f445751e640 1 -- 172.21.15.60:0/479073807 shutdown_connections 2023-08-24T10:08:47.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.853+0000 7f445751e640 1 -- 172.21.15.60:0/479073807 wait complete. 2023-08-24T10:08:47.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.854+0000 7f445751e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:47.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.854+0000 7f445751e640 1 Processor -- start 2023-08-24T10:08:47.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.855+0000 7f445751e640 1 -- start start 2023-08-24T10:08:47.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.855+0000 7f445751e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f445012ece0 0x7f445012b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:47.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.855+0000 7f445751e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445012b140 con 0x7f445012ece0 2023-08-24T10:08:47.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.855+0000 7f44552ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f445012ece0 0x7f445012b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:47.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.855+0000 7f44552ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f445012ece0 0x7f445012b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33444/0 (socket says 172.21.15.60:33444) 2023-08-24T10:08:47.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.855+0000 7f44552ba640 1 -- 172.21.15.60:0/1010534900 learned_addr learned my addr 172.21.15.60:0/1010534900 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:47.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.856+0000 7f44552ba640 1 -- 172.21.15.60:0/1010534900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4450089380 con 0x7f445012ece0 2023-08-24T10:08:47.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.856+0000 7f44552ba640 1 --2- 172.21.15.60:0/1010534900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f445012ece0 0x7f445012b500 crc :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:47.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.856+0000 7f444dffb640 1 -- 172.21.15.60:0/1010534900 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f444800f040 con 0x7f445012ece0 2023-08-24T10:08:47.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.856+0000 7f444dffb640 1 -- 172.21.15.60:0/1010534900 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f444800a5e0 con 0x7f445012ece0 2023-08-24T10:08:47.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.856+0000 7f444dffb640 1 -- 172.21.15.60:0/1010534900 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4448007a30 con 0x7f445012ece0 2023-08-24T10:08:47.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.856+0000 7f445751e640 1 -- 172.21.15.60:0/1010534900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445008b810 con 0x7f445012ece0 2023-08-24T10:08:47.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.856+0000 7f445751e640 1 -- 172.21.15.60:0/1010534900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445008a110 con 0x7f445012ece0 2023-08-24T10:08:47.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.857+0000 7f444dffb640 1 -- 172.21.15.60:0/1010534900 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f4448007b90 con 0x7f445012ece0 2023-08-24T10:08:47.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.857+0000 7f444dffb640 1 --2- 172.21.15.60:0/1010534900 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f443c037130 0x7f443c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:47.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.857+0000 7f444dffb640 1 -- 172.21.15.60:0/1010534900 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f4448047fc0 con 0x7f445012ece0 2023-08-24T10:08:47.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.857+0000 7f445751e640 1 -- 172.21.15.60:0/1010534900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4450096e80 con 0x7f445012ece0 2023-08-24T10:08:47.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.857+0000 7f4454ab9640 1 --2- 172.21.15.60:0/1010534900 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f443c037130 0x7f443c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:47.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.858+0000 7f4454ab9640 1 --2- 172.21.15.60:0/1010534900 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f443c037130 0x7f443c039640 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:47.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:47.862+0000 7f444dffb640 1 -- 172.21.15.60:0/1010534900 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f4448011690 con 0x7f445012ece0 2023-08-24T10:08:48.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.105+0000 7f445751e640 1 -- 172.21.15.60:0/1010534900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f4450061800 con 0x7f445012ece0 2023-08-24T10:08:48.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.105+0000 7f444dffb640 1 -- 172.21.15.60:0/1010534900 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f4448011280 con 0x7f445012ece0 2023-08-24T10:08:48.108 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:48.108 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":238,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:08:48.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.107+0000 7f445751e640 1 -- 172.21.15.60:0/1010534900 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f443c037130 msgr2=0x7f443c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:48.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.107+0000 7f445751e640 1 --2- 172.21.15.60:0/1010534900 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f443c037130 0x7f443c039640 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:48.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.107+0000 7f445751e640 1 -- 172.21.15.60:0/1010534900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f445012ece0 msgr2=0x7f445012b500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:48.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.107+0000 7f445751e640 1 --2- 172.21.15.60:0/1010534900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f445012ece0 0x7f445012b500 crc :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:48.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.107+0000 7f445751e640 1 -- 172.21.15.60:0/1010534900 shutdown_connections 2023-08-24T10:08:48.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.107+0000 7f445751e640 1 --2- 172.21.15.60:0/1010534900 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f443c037130 0x7f443c039640 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:48.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.107+0000 7f445751e640 1 --2- 172.21.15.60:0/1010534900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f445012ece0 0x7f445012b500 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:48.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.107+0000 7f445751e640 1 -- 172.21.15.60:0/1010534900 >> 172.21.15.60:0/1010534900 conn(0x7f445009e110 msgr2=0x7f4450097710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:48.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.107+0000 7f445751e640 1 -- 172.21.15.60:0/1010534900 shutdown_connections 2023-08-24T10:08:48.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.107+0000 7f445751e640 1 -- 172.21.15.60:0/1010534900 wait complete. 2023-08-24T10:08:48.124 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:08:48.124 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:48.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.232+0000 7f4101a63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:48.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.236+0000 7f4101a63640 1 Processor -- start 2023-08-24T10:08:48.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.236+0000 7f4101a63640 1 -- start start 2023-08-24T10:08:48.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.236+0000 7f4101a63640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f40fc132b50 0x7f40fc132f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:48.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.236+0000 7f4101a63640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40fc132610 con 0x7f40fc12f110 2023-08-24T10:08:48.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.236+0000 7f4101a63640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40fc09c260 con 0x7f40fc132b50 2023-08-24T10:08:48.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.236+0000 7f40fb7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f40fc132b50 0x7f40fc132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:48.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.236+0000 7f40fb7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f40fc132b50 0x7f40fc132f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33454/0 (socket says 172.21.15.60:33454) 2023-08-24T10:08:48.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.236+0000 7f40faffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f40fc12f110 0x7f40fc12f520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57350/0 (socket says 172.21.15.60:57350) 2023-08-24T10:08:48.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.236+0000 7f40fb7fe640 1 -- 172.21.15.60:0/2490139939 learned_addr learned my addr 172.21.15.60:0/2490139939 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:48.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.237+0000 7f40fb7fe640 1 -- 172.21.15.60:0/2490139939 >> v1:172.21.15.60:6789/0 conn(0x7f40fc12f110 legacy=0x7f40fc12f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:48.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.237+0000 7f40fb7fe640 1 -- 172.21.15.60:0/2490139939 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc132840 con 0x7f40fc132b50 2023-08-24T10:08:48.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.237+0000 7f40fb7fe640 1 --2- 172.21.15.60:0/2490139939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40fc132b50 0x7f40fc132f60 crc :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dea987c31da3704 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:48.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.237+0000 7f40fa7fc640 1 -- 172.21.15.60:0/2490139939 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f40ec009dd0 con 0x7f40fc132b50 2023-08-24T10:08:48.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.237+0000 7f40fa7fc640 1 -- 172.21.15.60:0/2490139939 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f40ec00e230 con 0x7f40fc132b50 2023-08-24T10:08:48.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.237+0000 7f40fa7fc640 1 -- 172.21.15.60:0/2490139939 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f40ec00e4d0 con 0x7f40fc132b50 2023-08-24T10:08:48.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.238+0000 7f4101a63640 1 -- 172.21.15.60:0/2490139939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40fc132b50 msgr2=0x7f40fc132f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:48.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.238+0000 7f4101a63640 1 --2- 172.21.15.60:0/2490139939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40fc132b50 0x7f40fc132f60 crc :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:48.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.238+0000 7f4101a63640 1 -- 172.21.15.60:0/2490139939 shutdown_connections 2023-08-24T10:08:48.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.238+0000 7f4101a63640 1 --2- 172.21.15.60:0/2490139939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40fc132b50 0x7f40fc132f60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:48.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.238+0000 7f4101a63640 1 -- 172.21.15.60:0/2490139939 >> 172.21.15.60:0/2490139939 conn(0x7f40fc0a26a0 msgr2=0x7f40fc0a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:48.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.238+0000 7f4101a63640 1 -- 172.21.15.60:0/2490139939 shutdown_connections 2023-08-24T10:08:48.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.238+0000 7f4101a63640 1 -- 172.21.15.60:0/2490139939 wait complete. 2023-08-24T10:08:48.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.238+0000 7f4101a63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:48.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.239+0000 7f4101a63640 1 Processor -- start 2023-08-24T10:08:48.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.239+0000 7f4101a63640 1 -- start start 2023-08-24T10:08:48.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.240+0000 7f4101a63640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40fc12f110 0x7f40fc09fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:48.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.240+0000 7f4101a63640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc12cc80 con 0x7f40fc12f110 2023-08-24T10:08:48.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.240+0000 7f40fb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40fc12f110 0x7f40fc09fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:48.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.240+0000 7f40fb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40fc12f110 0x7f40fc09fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33466/0 (socket says 172.21.15.60:33466) 2023-08-24T10:08:48.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.240+0000 7f40fb7fe640 1 -- 172.21.15.60:0/3451343048 learned_addr learned my addr 172.21.15.60:0/3451343048 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:48.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.240+0000 7f40fb7fe640 1 -- 172.21.15.60:0/3451343048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc09f400 con 0x7f40fc12f110 2023-08-24T10:08:48.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.240+0000 7f40fb7fe640 1 --2- 172.21.15.60:0/3451343048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40fc12f110 0x7f40fc09fcd0 crc :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:48.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.240+0000 7f40e3fff640 1 -- 172.21.15.60:0/3451343048 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f40ec014410 con 0x7f40fc12f110 2023-08-24T10:08:48.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.240+0000 7f40e3fff640 1 -- 172.21.15.60:0/3451343048 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f40ec014570 con 0x7f40fc12f110 2023-08-24T10:08:48.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.240+0000 7f40e3fff640 1 -- 172.21.15.60:0/3451343048 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f40ec014810 con 0x7f40fc12f110 2023-08-24T10:08:48.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.240+0000 7f4101a63640 1 -- 172.21.15.60:0/3451343048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40fc0a18e0 con 0x7f40fc12f110 2023-08-24T10:08:48.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.241+0000 7f4101a63640 1 -- 172.21.15.60:0/3451343048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40fc06a1d0 con 0x7f40fc12f110 2023-08-24T10:08:48.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.241+0000 7f40e3fff640 1 -- 172.21.15.60:0/3451343048 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f40ec0149c0 con 0x7f40fc12f110 2023-08-24T10:08:48.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.241+0000 7f4101a63640 1 -- 172.21.15.60:0/3451343048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40fc11c2b0 con 0x7f40fc12f110 2023-08-24T10:08:48.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.241+0000 7f40e3fff640 1 --2- 172.21.15.60:0/3451343048 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f40e8037130 0x7f40e8039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:48.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.242+0000 7f40e3fff640 1 -- 172.21.15.60:0/3451343048 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f40ec019040 con 0x7f40fc12f110 2023-08-24T10:08:48.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.242+0000 7f40faffd640 1 --2- 172.21.15.60:0/3451343048 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f40e8037130 0x7f40e8039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:48.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.242+0000 7f40faffd640 1 --2- 172.21.15.60:0/3451343048 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f40e8037130 0x7f40e8039640 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:48.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.246+0000 7f40e3fff640 1 -- 172.21.15.60:0/3451343048 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f40ec00f8f0 con 0x7f40fc12f110 2023-08-24T10:08:48.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.423+0000 7f4101a63640 1 -- 172.21.15.60:0/3451343048 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40fc1334a0 con 0x7f40e8037130 2023-08-24T10:08:48.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.424+0000 7f40e3fff640 1 -- 172.21.15.60:0/3451343048 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f40f00070b0 con 0x7f40e8037130 2023-08-24T10:08:48.426 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:48.426 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:48.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.426+0000 7f4101a63640 1 -- 172.21.15.60:0/3451343048 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f40e8037130 msgr2=0x7f40e8039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:48.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.426+0000 7f4101a63640 1 --2- 172.21.15.60:0/3451343048 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f40e8037130 0x7f40e8039640 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:48.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.426+0000 7f4101a63640 1 -- 172.21.15.60:0/3451343048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40fc12f110 msgr2=0x7f40fc09fcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:48.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.426+0000 7f4101a63640 1 --2- 172.21.15.60:0/3451343048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40fc12f110 0x7f40fc09fcd0 crc :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:48.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.426+0000 7f4101a63640 1 -- 172.21.15.60:0/3451343048 shutdown_connections 2023-08-24T10:08:48.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.426+0000 7f4101a63640 1 --2- 172.21.15.60:0/3451343048 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f40e8037130 0x7f40e8039640 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:48.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.426+0000 7f4101a63640 1 --2- 172.21.15.60:0/3451343048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40fc12f110 0x7f40fc09fcd0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:48.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.426+0000 7f4101a63640 1 -- 172.21.15.60:0/3451343048 >> 172.21.15.60:0/3451343048 conn(0x7f40fc0a26a0 msgr2=0x7f40fc1ba920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:48.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.426+0000 7f4101a63640 1 -- 172.21.15.60:0/3451343048 shutdown_connections 2023-08-24T10:08:48.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:48.426+0000 7f4101a63640 1 -- 172.21.15.60:0/3451343048 wait complete. 2023-08-24T10:08:48.438 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2023-08-24T10:08:47.539020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:51.213 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~224s 2023-08-24T10:08:51.213 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~224s 2023-08-24T10:08:51.213 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~224s 2023-08-24T10:08:51.214 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~224s 2023-08-24T10:08:51.440 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:51.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.561+0000 7fb1dcaac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:51.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.565+0000 7fb1dcaac640 1 Processor -- start 2023-08-24T10:08:51.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.565+0000 7fb1dcaac640 1 -- start start 2023-08-24T10:08:51.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.565+0000 7fb1dcaac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb1d806d300 0x7fb1d812bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:51.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.565+0000 7fb1dcaac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d8099440 con 0x7fb1d812c1c0 2023-08-24T10:08:51.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.565+0000 7fb1dcaac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d809a050 con 0x7fb1d806d300 2023-08-24T10:08:51.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.566+0000 7fb1d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb1d806d300 0x7fb1d812bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:51.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.566+0000 7fb1d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb1d806d300 0x7fb1d812bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33482/0 (socket says 172.21.15.60:33482) 2023-08-24T10:08:51.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.566+0000 7fb1d659c640 1 -- 172.21.15.60:0/3099614957 learned_addr learned my addr 172.21.15.60:0/3099614957 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:51.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.566+0000 7fb1d5d9b640 1 --1- 172.21.15.60:0/3099614957 >> v1:172.21.15.60:6789/0 conn(0x7fb1d812c1c0 0x7fb1d812ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57364/0 (socket says 172.21.15.60:57364) 2023-08-24T10:08:51.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.566+0000 7fb1d659c640 1 -- 172.21.15.60:0/3099614957 >> v1:172.21.15.60:6789/0 conn(0x7fb1d812c1c0 legacy=0x7fb1d812ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:51.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.566+0000 7fb1d659c640 1 -- 172.21.15.60:0/3099614957 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d812b9d0 con 0x7fb1d806d300 2023-08-24T10:08:51.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.566+0000 7fb1d659c640 1 --2- 172.21.15.60:0/3099614957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d806d300 0x7fb1d812bc80 crc :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbb9313947e2d1f server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:51.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.566+0000 7fb1d559a640 1 -- 172.21.15.60:0/3099614957 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb1c8009dd0 con 0x7fb1d806d300 2023-08-24T10:08:51.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.566+0000 7fb1d559a640 1 -- 172.21.15.60:0/3099614957 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb1c800e230 con 0x7fb1d806d300 2023-08-24T10:08:51.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.566+0000 7fb1d559a640 1 -- 172.21.15.60:0/3099614957 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb1c800e4d0 con 0x7fb1d806d300 2023-08-24T10:08:51.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.567+0000 7fb1dcaac640 1 -- 172.21.15.60:0/3099614957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d806d300 msgr2=0x7fb1d812bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:51.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.567+0000 7fb1dcaac640 1 --2- 172.21.15.60:0/3099614957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d806d300 0x7fb1d812bc80 crc :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:51.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.567+0000 7fb1dcaac640 1 -- 172.21.15.60:0/3099614957 shutdown_connections 2023-08-24T10:08:51.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.567+0000 7fb1dcaac640 1 --2- 172.21.15.60:0/3099614957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d806d300 0x7fb1d812bc80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:51.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.567+0000 7fb1dcaac640 1 -- 172.21.15.60:0/3099614957 >> 172.21.15.60:0/3099614957 conn(0x7fb1d809e110 msgr2=0x7fb1d809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:51.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.567+0000 7fb1dcaac640 1 -- 172.21.15.60:0/3099614957 shutdown_connections 2023-08-24T10:08:51.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.567+0000 7fb1dcaac640 1 -- 172.21.15.60:0/3099614957 wait complete. 2023-08-24T10:08:51.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.567+0000 7fb1dcaac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:51.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.568+0000 7fb1dcaac640 1 Processor -- start 2023-08-24T10:08:51.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.568+0000 7fb1dcaac640 1 -- start start 2023-08-24T10:08:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.569+0000 7fb1dcaac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d806d300 0x7fb1d812c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.569+0000 7fb1dcaac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d8096d00 con 0x7fb1d806d300 2023-08-24T10:08:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.569+0000 7fb1d659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d806d300 0x7fb1d812c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.569+0000 7fb1d659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d806d300 0x7fb1d812c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33498/0 (socket says 172.21.15.60:33498) 2023-08-24T10:08:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.569+0000 7fb1d659c640 1 -- 172.21.15.60:0/4290056207 learned_addr learned my addr 172.21.15.60:0/4290056207 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:51.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.570+0000 7fb1d659c640 1 -- 172.21.15.60:0/4290056207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d8070dc0 con 0x7fb1d806d300 2023-08-24T10:08:51.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.570+0000 7fb1d659c640 1 --2- 172.21.15.60:0/4290056207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d806d300 0x7fb1d812c910 crc :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:51.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.570+0000 7fb1c6ffd640 1 -- 172.21.15.60:0/4290056207 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb1c8014540 con 0x7fb1d806d300 2023-08-24T10:08:51.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.570+0000 7fb1c6ffd640 1 -- 172.21.15.60:0/4290056207 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb1c80146a0 con 0x7fb1d806d300 2023-08-24T10:08:51.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.570+0000 7fb1c6ffd640 1 -- 172.21.15.60:0/4290056207 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb1c8014940 con 0x7fb1d806d300 2023-08-24T10:08:51.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.570+0000 7fb1dcaac640 1 -- 172.21.15.60:0/4290056207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d8096ef0 con 0x7fb1d806d300 2023-08-24T10:08:51.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.570+0000 7fb1dcaac640 1 -- 172.21.15.60:0/4290056207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d8071550 con 0x7fb1d806d300 2023-08-24T10:08:51.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.571+0000 7fb1c6ffd640 1 -- 172.21.15.60:0/4290056207 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fb1c8014af0 con 0x7fb1d806d300 2023-08-24T10:08:51.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.571+0000 7fb1dcaac640 1 -- 172.21.15.60:0/4290056207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d8061800 con 0x7fb1d806d300 2023-08-24T10:08:51.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.571+0000 7fb1c6ffd640 1 --2- 172.21.15.60:0/4290056207 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb1b8037130 0x7fb1b8039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:51.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.571+0000 7fb1c6ffd640 1 -- 172.21.15.60:0/4290056207 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fb1c8019040 con 0x7fb1d806d300 2023-08-24T10:08:51.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.571+0000 7fb1d5d9b640 1 --2- 172.21.15.60:0/4290056207 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb1b8037130 0x7fb1b8039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:51.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.572+0000 7fb1d5d9b640 1 --2- 172.21.15.60:0/4290056207 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb1b8037130 0x7fb1b8039640 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:51.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.576+0000 7fb1c6ffd640 1 -- 172.21.15.60:0/4290056207 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fb1c8018190 con 0x7fb1d806d300 2023-08-24T10:08:51.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.749+0000 7fb1dcaac640 1 -- 172.21.15.60:0/4290056207 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb1d808d640 con 0x7fb1b8037130 2023-08-24T10:08:51.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.750+0000 7fb1c6ffd640 1 -- 172.21.15.60:0/4290056207 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7fb1d808d640 con 0x7fb1b8037130 2023-08-24T10:08:51.752 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:51.752 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:51.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.753+0000 7fb1dcaac640 1 -- 172.21.15.60:0/4290056207 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb1b8037130 msgr2=0x7fb1b8039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:51.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.753+0000 7fb1dcaac640 1 --2- 172.21.15.60:0/4290056207 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb1b8037130 0x7fb1b8039640 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:51.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.753+0000 7fb1dcaac640 1 -- 172.21.15.60:0/4290056207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d806d300 msgr2=0x7fb1d812c910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:51.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.753+0000 7fb1dcaac640 1 --2- 172.21.15.60:0/4290056207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d806d300 0x7fb1d812c910 crc :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:51.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.753+0000 7fb1dcaac640 1 -- 172.21.15.60:0/4290056207 shutdown_connections 2023-08-24T10:08:51.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.753+0000 7fb1dcaac640 1 --2- 172.21.15.60:0/4290056207 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb1b8037130 0x7fb1b8039640 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:51.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.753+0000 7fb1dcaac640 1 --2- 172.21.15.60:0/4290056207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d806d300 0x7fb1d812c910 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:51.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.753+0000 7fb1dcaac640 1 -- 172.21.15.60:0/4290056207 >> 172.21.15.60:0/4290056207 conn(0x7fb1d809e110 msgr2=0x7fb1d81252d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:51.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.753+0000 7fb1dcaac640 1 -- 172.21.15.60:0/4290056207 shutdown_connections 2023-08-24T10:08:51.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.753+0000 7fb1dcaac640 1 -- 172.21.15.60:0/4290056207 wait complete. 2023-08-24T10:08:51.764 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2023-08-24T10:08:51.539968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:51.765 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:51.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.872+0000 7f926e2ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:51.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.876+0000 7f926e2ec640 1 Processor -- start 2023-08-24T10:08:51.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.876+0000 7f926e2ec640 1 -- start start 2023-08-24T10:08:51.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.876+0000 7f926e2ec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9268096860 0x7f9268096c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:51.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.876+0000 7f926e2ec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92680971b0 con 0x7f92680990b0 2023-08-24T10:08:51.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.876+0000 7f926e2ec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926806db70 con 0x7f9268096860 2023-08-24T10:08:51.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.877+0000 7f9267fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9268096860 0x7f9268096c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:51.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.877+0000 7f9267fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9268096860 0x7f9268096c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33506/0 (socket says 172.21.15.60:33506) 2023-08-24T10:08:51.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.877+0000 7f9267fff640 1 -- 172.21.15.60:0/2407463781 learned_addr learned my addr 172.21.15.60:0/2407463781 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:51.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.877+0000 7f9267fff640 1 -- 172.21.15.60:0/2407463781 >> v1:172.21.15.60:6789/0 conn(0x7f92680990b0 legacy=0x7f92680994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:51.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.877+0000 7f9267fff640 1 -- 172.21.15.60:0/2407463781 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926806e320 con 0x7f9268096860 2023-08-24T10:08:51.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.877+0000 7f9267fff640 1 --2- 172.21.15.60:0/2407463781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9268096860 0x7f9268096c70 crc :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9921144e30f672cf server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:51.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.877+0000 7f9266ffd640 1 -- 172.21.15.60:0/2407463781 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9258009dd0 con 0x7f9268096860 2023-08-24T10:08:51.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.877+0000 7f9266ffd640 1 -- 172.21.15.60:0/2407463781 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f925800e230 con 0x7f9268096860 2023-08-24T10:08:51.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.877+0000 7f9266ffd640 1 -- 172.21.15.60:0/2407463781 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f925800e4d0 con 0x7f9268096860 2023-08-24T10:08:51.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.878+0000 7f926e2ec640 1 -- 172.21.15.60:0/2407463781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9268096860 msgr2=0x7f9268096c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:51.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.878+0000 7f926e2ec640 1 --2- 172.21.15.60:0/2407463781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9268096860 0x7f9268096c70 crc :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:51.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.878+0000 7f926e2ec640 1 -- 172.21.15.60:0/2407463781 shutdown_connections 2023-08-24T10:08:51.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.878+0000 7f926e2ec640 1 --2- 172.21.15.60:0/2407463781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9268096860 0x7f9268096c70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:51.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.878+0000 7f926e2ec640 1 -- 172.21.15.60:0/2407463781 >> 172.21.15.60:0/2407463781 conn(0x7f926809e110 msgr2=0x7f926809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:51.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.878+0000 7f926e2ec640 1 -- 172.21.15.60:0/2407463781 shutdown_connections 2023-08-24T10:08:51.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.878+0000 7f926e2ec640 1 -- 172.21.15.60:0/2407463781 wait complete. 2023-08-24T10:08:51.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.878+0000 7f926e2ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:51.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.879+0000 7f926e2ec640 1 Processor -- start 2023-08-24T10:08:51.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.879+0000 7f926e2ec640 1 -- start start 2023-08-24T10:08:51.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.880+0000 7f926e2ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9268096860 0x7f9268135650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:51.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.880+0000 7f926e2ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9268136ff0 con 0x7f9268096860 2023-08-24T10:08:51.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.880+0000 7f9267fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9268096860 0x7f9268135650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:51.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.880+0000 7f9267fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9268096860 0x7f9268135650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33522/0 (socket says 172.21.15.60:33522) 2023-08-24T10:08:51.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.880+0000 7f9267fff640 1 -- 172.21.15.60:0/3492110481 learned_addr learned my addr 172.21.15.60:0/3492110481 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:51.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.880+0000 7f9267fff640 1 -- 172.21.15.60:0/3492110481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92681375e0 con 0x7f9268096860 2023-08-24T10:08:51.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.880+0000 7f9267fff640 1 --2- 172.21.15.60:0/3492110481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9268096860 0x7f9268135650 crc :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:51.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.881+0000 7f9264ff9640 1 -- 172.21.15.60:0/3492110481 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9258014410 con 0x7f9268096860 2023-08-24T10:08:51.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.881+0000 7f9264ff9640 1 -- 172.21.15.60:0/3492110481 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f9258014570 con 0x7f9268096860 2023-08-24T10:08:51.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.881+0000 7f926e2ec640 1 -- 172.21.15.60:0/3492110481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926808dd60 con 0x7f9268096860 2023-08-24T10:08:51.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.881+0000 7f9264ff9640 1 -- 172.21.15.60:0/3492110481 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9258014810 con 0x7f9268096860 2023-08-24T10:08:51.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.881+0000 7f926e2ec640 1 -- 172.21.15.60:0/3492110481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9268071630 con 0x7f9268096860 2023-08-24T10:08:51.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.882+0000 7f9264ff9640 1 -- 172.21.15.60:0/3492110481 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f92580149c0 con 0x7f9268096860 2023-08-24T10:08:51.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.882+0000 7f926e2ec640 1 -- 172.21.15.60:0/3492110481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926806ceb0 con 0x7f9268096860 2023-08-24T10:08:51.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.882+0000 7f9264ff9640 1 --2- 172.21.15.60:0/3492110481 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9254037130 0x7f9254039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:51.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.882+0000 7f9264ff9640 1 -- 172.21.15.60:0/3492110481 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f9258019040 con 0x7f9268096860 2023-08-24T10:08:51.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.882+0000 7f92677fe640 1 --2- 172.21.15.60:0/3492110481 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9254037130 0x7f9254039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:51.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.883+0000 7f92677fe640 1 --2- 172.21.15.60:0/3492110481 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9254037130 0x7f9254039640 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:51.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:51.886+0000 7f9264ff9640 1 -- 172.21.15.60:0/3492110481 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f925800f8f0 con 0x7f9268096860 2023-08-24T10:08:52.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.058+0000 7f926e2ec640 1 -- 172.21.15.60:0/3492110481 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f926806e5a0 con 0x7f9254037130 2023-08-24T10:08:52.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.059+0000 7f9264ff9640 1 -- 172.21.15.60:0/3492110481 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f925c001650 con 0x7f9254037130 2023-08-24T10:08:52.061 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:52.062 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:52.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.061+0000 7f926e2ec640 1 -- 172.21.15.60:0/3492110481 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9254037130 msgr2=0x7f9254039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:52.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.061+0000 7f926e2ec640 1 --2- 172.21.15.60:0/3492110481 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9254037130 0x7f9254039640 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:52.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.061+0000 7f926e2ec640 1 -- 172.21.15.60:0/3492110481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9268096860 msgr2=0x7f9268135650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:52.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.061+0000 7f926e2ec640 1 --2- 172.21.15.60:0/3492110481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9268096860 0x7f9268135650 crc :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:52.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.061+0000 7f926e2ec640 1 -- 172.21.15.60:0/3492110481 shutdown_connections 2023-08-24T10:08:52.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.061+0000 7f926e2ec640 1 --2- 172.21.15.60:0/3492110481 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9254037130 0x7f9254039640 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:52.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.061+0000 7f926e2ec640 1 --2- 172.21.15.60:0/3492110481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9268096860 0x7f9268135650 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:52.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.061+0000 7f926e2ec640 1 -- 172.21.15.60:0/3492110481 >> 172.21.15.60:0/3492110481 conn(0x7f926809e110 msgr2=0x7f926809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:52.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.061+0000 7f926e2ec640 1 -- 172.21.15.60:0/3492110481 shutdown_connections 2023-08-24T10:08:52.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.061+0000 7f926e2ec640 1 -- 172.21.15.60:0/3492110481 wait complete. 2023-08-24T10:08:52.072 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2023-08-24T10:08:51.539968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:52.073 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:08:52.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:52.074 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:08:52.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:52.074 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:08:52.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:52.075 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:08:52.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:52.075 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:08:52.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:52.076 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:08:52.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:52.076 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:08:52.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:52.077 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:08:52.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:52.077 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:08:52.078 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:52.078 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:08:52.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.185+0000 7fb697d9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:52.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.188+0000 7fb697d9e640 1 Processor -- start 2023-08-24T10:08:52.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.188+0000 7fb697d9e640 1 -- start start 2023-08-24T10:08:52.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.189+0000 7fb697d9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb69012f610 0x7fb69012fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:52.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.189+0000 7fb697d9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb69009d750 con 0x7fb6901326f0 2023-08-24T10:08:52.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.189+0000 7fb697d9e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69009d980 con 0x7fb69012f610 2023-08-24T10:08:52.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.189+0000 7fb695b3a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb69012f610 0x7fb69012fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:52.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.189+0000 7fb695b3a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb69012f610 0x7fb69012fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33534/0 (socket says 172.21.15.60:33534) 2023-08-24T10:08:52.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.189+0000 7fb695339640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb6901326f0 0x7fb690132b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57382/0 (socket says 172.21.15.60:57382) 2023-08-24T10:08:52.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.189+0000 7fb695b3a640 1 -- 172.21.15.60:0/1518409317 learned_addr learned my addr 172.21.15.60:0/1518409317 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:52.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.190+0000 7fb695b3a640 1 -- 172.21.15.60:0/1518409317 >> v1:172.21.15.60:6789/0 conn(0x7fb6901326f0 legacy=0x7fb690132b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:52.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.190+0000 7fb695b3a640 1 -- 172.21.15.60:0/1518409317 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69009bde0 con 0x7fb69012f610 2023-08-24T10:08:52.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.190+0000 7fb695b3a640 1 --2- 172.21.15.60:0/1518409317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69012f610 0x7fb69012fa20 crc :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2eba2dc2c77627e server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:52.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.190+0000 7fb694b38640 1 -- 172.21.15.60:0/1518409317 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb688009dc0 con 0x7fb69012f610 2023-08-24T10:08:52.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.190+0000 7fb694b38640 1 -- 172.21.15.60:0/1518409317 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb68800e230 con 0x7fb69012f610 2023-08-24T10:08:52.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.190+0000 7fb694b38640 1 -- 172.21.15.60:0/1518409317 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb68800e4d0 con 0x7fb69012f610 2023-08-24T10:08:52.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.190+0000 7fb697d9e640 1 -- 172.21.15.60:0/1518409317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69012f610 msgr2=0x7fb69012fa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:52.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.190+0000 7fb697d9e640 1 --2- 172.21.15.60:0/1518409317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69012f610 0x7fb69012fa20 crc :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:52.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.191+0000 7fb697d9e640 1 -- 172.21.15.60:0/1518409317 shutdown_connections 2023-08-24T10:08:52.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.191+0000 7fb697d9e640 1 --2- 172.21.15.60:0/1518409317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69012f610 0x7fb69012fa20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:52.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.191+0000 7fb697d9e640 1 -- 172.21.15.60:0/1518409317 >> 172.21.15.60:0/1518409317 conn(0x7fb6900a2500 msgr2=0x7fb6900a2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:52.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.191+0000 7fb697d9e640 1 -- 172.21.15.60:0/1518409317 shutdown_connections 2023-08-24T10:08:52.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.191+0000 7fb697d9e640 1 -- 172.21.15.60:0/1518409317 wait complete. 2023-08-24T10:08:52.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.191+0000 7fb697d9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:52.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.192+0000 7fb697d9e640 1 Processor -- start 2023-08-24T10:08:52.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.192+0000 7fb697d9e640 1 -- start start 2023-08-24T10:08:52.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.192+0000 7fb697d9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69012f610 0x7fb69009fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:52.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.192+0000 7fb697d9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69012cc60 con 0x7fb69012f610 2023-08-24T10:08:52.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.193+0000 7fb695b3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69012f610 0x7fb69009fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:52.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.193+0000 7fb695b3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69012f610 0x7fb69009fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33546/0 (socket says 172.21.15.60:33546) 2023-08-24T10:08:52.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.193+0000 7fb695b3a640 1 -- 172.21.15.60:0/1850369653 learned_addr learned my addr 172.21.15.60:0/1850369653 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:52.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.193+0000 7fb695b3a640 1 -- 172.21.15.60:0/1850369653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69006a1f0 con 0x7fb69012f610 2023-08-24T10:08:52.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.193+0000 7fb695b3a640 1 --2- 172.21.15.60:0/1850369653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69012f610 0x7fb69009fca0 crc :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:52.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.193+0000 7fb68e7fc640 1 -- 172.21.15.60:0/1850369653 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb688014540 con 0x7fb69012f610 2023-08-24T10:08:52.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.193+0000 7fb68e7fc640 1 -- 172.21.15.60:0/1850369653 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb6880146a0 con 0x7fb69012f610 2023-08-24T10:08:52.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.193+0000 7fb68e7fc640 1 -- 172.21.15.60:0/1850369653 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb688014940 con 0x7fb69012f610 2023-08-24T10:08:52.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.193+0000 7fb697d9e640 1 -- 172.21.15.60:0/1850369653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69009f7e0 con 0x7fb69012f610 2023-08-24T10:08:52.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.193+0000 7fb697d9e640 1 -- 172.21.15.60:0/1850369653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69009f4d0 con 0x7fb69012f610 2023-08-24T10:08:52.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.194+0000 7fb68e7fc640 1 -- 172.21.15.60:0/1850369653 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fb688014af0 con 0x7fb69012f610 2023-08-24T10:08:52.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.194+0000 7fb697d9e640 1 -- 172.21.15.60:0/1850369653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb690061800 con 0x7fb69012f610 2023-08-24T10:08:52.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.194+0000 7fb68e7fc640 1 --2- 172.21.15.60:0/1850369653 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb67c037130 0x7fb67c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:52.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.194+0000 7fb68e7fc640 1 -- 172.21.15.60:0/1850369653 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fb688019040 con 0x7fb69012f610 2023-08-24T10:08:52.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.195+0000 7fb695339640 1 --2- 172.21.15.60:0/1850369653 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb67c037130 0x7fb67c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:52.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.195+0000 7fb695339640 1 --2- 172.21.15.60:0/1850369653 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb67c037130 0x7fb67c039640 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:52.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.199+0000 7fb68e7fc640 1 -- 172.21.15.60:0/1850369653 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fb68800fb00 con 0x7fb69012f610 2023-08-24T10:08:52.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.442+0000 7fb697d9e640 1 -- 172.21.15.60:0/1850369653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb69006a1f0 con 0x7fb69012f610 2023-08-24T10:08:52.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.443+0000 7fb68e7fc640 1 -- 172.21.15.60:0/1850369653 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7fb69006a1f0 con 0x7fb69012f610 2023-08-24T10:08:52.445 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:52.445 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":243,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:08:52.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.445+0000 7fb697d9e640 1 -- 172.21.15.60:0/1850369653 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb67c037130 msgr2=0x7fb67c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:52.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.445+0000 7fb697d9e640 1 --2- 172.21.15.60:0/1850369653 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb67c037130 0x7fb67c039640 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:52.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.445+0000 7fb697d9e640 1 -- 172.21.15.60:0/1850369653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69012f610 msgr2=0x7fb69009fca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:52.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.445+0000 7fb697d9e640 1 --2- 172.21.15.60:0/1850369653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69012f610 0x7fb69009fca0 crc :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:52.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.445+0000 7fb697d9e640 1 -- 172.21.15.60:0/1850369653 shutdown_connections 2023-08-24T10:08:52.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.445+0000 7fb697d9e640 1 --2- 172.21.15.60:0/1850369653 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb67c037130 0x7fb67c039640 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:52.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.445+0000 7fb697d9e640 1 --2- 172.21.15.60:0/1850369653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb69012f610 0x7fb69009fca0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:52.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.445+0000 7fb697d9e640 1 -- 172.21.15.60:0/1850369653 >> 172.21.15.60:0/1850369653 conn(0x7fb6900a2500 msgr2=0x7fb6901ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:52.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.445+0000 7fb697d9e640 1 -- 172.21.15.60:0/1850369653 shutdown_connections 2023-08-24T10:08:52.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.445+0000 7fb697d9e640 1 -- 172.21.15.60:0/1850369653 wait complete. 2023-08-24T10:08:52.456 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:08:52.457 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:52.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.563+0000 7fb18019e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:52.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.567+0000 7fb18019e640 1 Processor -- start 2023-08-24T10:08:52.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.567+0000 7fb18019e640 1 -- start start 2023-08-24T10:08:52.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.568+0000 7fb18019e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb178132b50 0x7fb178132f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:52.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.568+0000 7fb18019e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb178132610 con 0x7fb17812f110 2023-08-24T10:08:52.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.568+0000 7fb18019e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb17809c260 con 0x7fb178132b50 2023-08-24T10:08:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.568+0000 7fb17df3a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb178132b50 0x7fb178132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.568+0000 7fb17df3a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb178132b50 0x7fb178132f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33556/0 (socket says 172.21.15.60:33556) 2023-08-24T10:08:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.568+0000 7fb17df3a640 1 -- 172.21.15.60:0/2731188944 learned_addr learned my addr 172.21.15.60:0/2731188944 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.569+0000 7fb17df3a640 1 -- 172.21.15.60:0/2731188944 >> v1:172.21.15.60:6789/0 conn(0x7fb17812f110 legacy=0x7fb17812f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.569+0000 7fb17df3a640 1 -- 172.21.15.60:0/2731188944 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb178132840 con 0x7fb178132b50 2023-08-24T10:08:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.569+0000 7fb17df3a640 1 --2- 172.21.15.60:0/2731188944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb178132b50 0x7fb178132f60 crc :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206172efb9fc6c3e server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.569+0000 7fb17cf38640 1 -- 172.21.15.60:0/2731188944 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb170009dd0 con 0x7fb178132b50 2023-08-24T10:08:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.569+0000 7fb17cf38640 1 -- 172.21.15.60:0/2731188944 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb17000e230 con 0x7fb178132b50 2023-08-24T10:08:52.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.569+0000 7fb17cf38640 1 -- 172.21.15.60:0/2731188944 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb17000e4d0 con 0x7fb178132b50 2023-08-24T10:08:52.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.570+0000 7fb18019e640 1 -- 172.21.15.60:0/2731188944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb178132b50 msgr2=0x7fb178132f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:52.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.570+0000 7fb18019e640 1 --2- 172.21.15.60:0/2731188944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb178132b50 0x7fb178132f60 crc :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:52.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.570+0000 7fb18019e640 1 -- 172.21.15.60:0/2731188944 shutdown_connections 2023-08-24T10:08:52.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.570+0000 7fb18019e640 1 --2- 172.21.15.60:0/2731188944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb178132b50 0x7fb178132f60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:52.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.570+0000 7fb18019e640 1 -- 172.21.15.60:0/2731188944 >> 172.21.15.60:0/2731188944 conn(0x7fb1780a26a0 msgr2=0x7fb1780a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:52.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.570+0000 7fb18019e640 1 -- 172.21.15.60:0/2731188944 shutdown_connections 2023-08-24T10:08:52.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.570+0000 7fb18019e640 1 -- 172.21.15.60:0/2731188944 wait complete. 2023-08-24T10:08:52.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.570+0000 7fb18019e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:52.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.571+0000 7fb18019e640 1 Processor -- start 2023-08-24T10:08:52.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.571+0000 7fb18019e640 1 -- start start 2023-08-24T10:08:52.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.572+0000 7fb18019e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb17812f110 0x7fb17809fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:52.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.572+0000 7fb18019e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17812cc80 con 0x7fb17812f110 2023-08-24T10:08:52.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.572+0000 7fb17df3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb17812f110 0x7fb17809fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:52.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.572+0000 7fb17df3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb17812f110 0x7fb17809fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33564/0 (socket says 172.21.15.60:33564) 2023-08-24T10:08:52.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.572+0000 7fb17df3a640 1 -- 172.21.15.60:0/3163244774 learned_addr learned my addr 172.21.15.60:0/3163244774 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:52.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.572+0000 7fb17df3a640 1 -- 172.21.15.60:0/3163244774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17809f400 con 0x7fb17812f110 2023-08-24T10:08:52.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.572+0000 7fb17df3a640 1 --2- 172.21.15.60:0/3163244774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb17812f110 0x7fb17809fcd0 crc :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:52.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.572+0000 7fb1767fc640 1 -- 172.21.15.60:0/3163244774 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb170014410 con 0x7fb17812f110 2023-08-24T10:08:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.572+0000 7fb1767fc640 1 -- 172.21.15.60:0/3163244774 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb170014570 con 0x7fb17812f110 2023-08-24T10:08:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.572+0000 7fb1767fc640 1 -- 172.21.15.60:0/3163244774 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb170014810 con 0x7fb17812f110 2023-08-24T10:08:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.572+0000 7fb18019e640 1 -- 172.21.15.60:0/3163244774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1780a18e0 con 0x7fb17812f110 2023-08-24T10:08:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.572+0000 7fb18019e640 1 -- 172.21.15.60:0/3163244774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17806a1d0 con 0x7fb17812f110 2023-08-24T10:08:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.573+0000 7fb1767fc640 1 -- 172.21.15.60:0/3163244774 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fb1700149c0 con 0x7fb17812f110 2023-08-24T10:08:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.573+0000 7fb18019e640 1 -- 172.21.15.60:0/3163244774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17811c2b0 con 0x7fb17812f110 2023-08-24T10:08:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.573+0000 7fb1767fc640 1 --2- 172.21.15.60:0/3163244774 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb164037130 0x7fb164039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.573+0000 7fb1767fc640 1 -- 172.21.15.60:0/3163244774 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fb170019040 con 0x7fb17812f110 2023-08-24T10:08:52.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.573+0000 7fb17d739640 1 --2- 172.21.15.60:0/3163244774 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb164037130 0x7fb164039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:52.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.574+0000 7fb17d739640 1 --2- 172.21.15.60:0/3163244774 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb164037130 0x7fb164039640 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:52.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.578+0000 7fb1767fc640 1 -- 172.21.15.60:0/3163244774 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fb17000f8f0 con 0x7fb17812f110 2023-08-24T10:08:52.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.749+0000 7fb18019e640 1 -- 172.21.15.60:0/3163244774 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb1781334a0 con 0x7fb164037130 2023-08-24T10:08:52.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.750+0000 7fb1767fc640 1 -- 172.21.15.60:0/3163244774 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7fb16c001650 con 0x7fb164037130 2023-08-24T10:08:52.752 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:52.752 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:52.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.752+0000 7fb18019e640 1 -- 172.21.15.60:0/3163244774 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb164037130 msgr2=0x7fb164039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:52.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.752+0000 7fb18019e640 1 --2- 172.21.15.60:0/3163244774 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb164037130 0x7fb164039640 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:52.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.752+0000 7fb18019e640 1 -- 172.21.15.60:0/3163244774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb17812f110 msgr2=0x7fb17809fcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:52.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.752+0000 7fb18019e640 1 --2- 172.21.15.60:0/3163244774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb17812f110 0x7fb17809fcd0 crc :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:52.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.752+0000 7fb18019e640 1 -- 172.21.15.60:0/3163244774 shutdown_connections 2023-08-24T10:08:52.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.752+0000 7fb18019e640 1 --2- 172.21.15.60:0/3163244774 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fb164037130 0x7fb164039640 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:52.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.752+0000 7fb18019e640 1 --2- 172.21.15.60:0/3163244774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb17812f110 0x7fb17809fcd0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:52.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.752+0000 7fb18019e640 1 -- 172.21.15.60:0/3163244774 >> 172.21.15.60:0/3163244774 conn(0x7fb1780a26a0 msgr2=0x7fb1781ba920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:52.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.752+0000 7fb18019e640 1 -- 172.21.15.60:0/3163244774 shutdown_connections 2023-08-24T10:08:52.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:52.752+0000 7fb18019e640 1 -- 172.21.15.60:0/3163244774 wait complete. 2023-08-24T10:08:52.763 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2023-08-24T10:08:51.539968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:55.766 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:55.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.890+0000 7f4eb9aac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:55.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.894+0000 7f4eb9aac640 1 Processor -- start 2023-08-24T10:08:55.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.894+0000 7f4eb9aac640 1 -- start start 2023-08-24T10:08:55.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.895+0000 7f4eb9aac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4eb406d300 0x7f4eb412bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:55.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.895+0000 7f4eb9aac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb4099440 con 0x7f4eb412c1c0 2023-08-24T10:08:55.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.895+0000 7f4eb9aac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb409a050 con 0x7f4eb406d300 2023-08-24T10:08:55.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.895+0000 7f4eb37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4eb406d300 0x7f4eb412bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:55.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.895+0000 7f4eb37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4eb406d300 0x7f4eb412bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33580/0 (socket says 172.21.15.60:33580) 2023-08-24T10:08:55.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.895+0000 7f4eb37fe640 1 -- 172.21.15.60:0/316645287 learned_addr learned my addr 172.21.15.60:0/316645287 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:55.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.896+0000 7f4eb37fe640 1 -- 172.21.15.60:0/316645287 >> v1:172.21.15.60:6789/0 conn(0x7f4eb412c1c0 legacy=0x7f4eb412ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:55.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.896+0000 7f4eb37fe640 1 -- 172.21.15.60:0/316645287 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb412b9d0 con 0x7f4eb406d300 2023-08-24T10:08:55.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.896+0000 7f4eb37fe640 1 --2- 172.21.15.60:0/316645287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb406d300 0x7f4eb412bc80 crc :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6723e8d6308881e6 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:55.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.896+0000 7f4eb27fc640 1 -- 172.21.15.60:0/316645287 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4ea4009dd0 con 0x7f4eb406d300 2023-08-24T10:08:55.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.896+0000 7f4eb27fc640 1 -- 172.21.15.60:0/316645287 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4ea400e230 con 0x7f4eb406d300 2023-08-24T10:08:55.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.896+0000 7f4eb27fc640 1 -- 172.21.15.60:0/316645287 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4ea400e4d0 con 0x7f4eb406d300 2023-08-24T10:08:55.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.897+0000 7f4eb9aac640 1 -- 172.21.15.60:0/316645287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb406d300 msgr2=0x7f4eb412bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:55.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.897+0000 7f4eb9aac640 1 --2- 172.21.15.60:0/316645287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb406d300 0x7f4eb412bc80 crc :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:55.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.897+0000 7f4eb9aac640 1 -- 172.21.15.60:0/316645287 shutdown_connections 2023-08-24T10:08:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.897+0000 7f4eb9aac640 1 --2- 172.21.15.60:0/316645287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb406d300 0x7f4eb412bc80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.897+0000 7f4eb9aac640 1 -- 172.21.15.60:0/316645287 >> 172.21.15.60:0/316645287 conn(0x7f4eb409e110 msgr2=0x7f4eb409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.897+0000 7f4eb9aac640 1 -- 172.21.15.60:0/316645287 shutdown_connections 2023-08-24T10:08:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.897+0000 7f4eb9aac640 1 -- 172.21.15.60:0/316645287 wait complete. 2023-08-24T10:08:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.897+0000 7f4eb9aac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.898+0000 7f4eb9aac640 1 Processor -- start 2023-08-24T10:08:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.898+0000 7f4eb9aac640 1 -- start start 2023-08-24T10:08:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.899+0000 7f4eb9aac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb406d300 0x7f4eb4096580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.899+0000 7f4eb9aac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb4071700 con 0x7f4eb406d300 2023-08-24T10:08:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.899+0000 7f4eb37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb406d300 0x7f4eb4096580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.899+0000 7f4eb37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb406d300 0x7f4eb4096580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33590/0 (socket says 172.21.15.60:33590) 2023-08-24T10:08:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.899+0000 7f4eb37fe640 1 -- 172.21.15.60:0/2988139334 learned_addr learned my addr 172.21.15.60:0/2988139334 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.899+0000 7f4eb37fe640 1 -- 172.21.15.60:0/2988139334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb40714d0 con 0x7f4eb406d300 2023-08-24T10:08:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.900+0000 7f4eb37fe640 1 --2- 172.21.15.60:0/2988139334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb406d300 0x7f4eb4096580 crc :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.900+0000 7f4e9bfff640 1 -- 172.21.15.60:0/2988139334 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4ea4014540 con 0x7f4eb406d300 2023-08-24T10:08:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.900+0000 7f4e9bfff640 1 -- 172.21.15.60:0/2988139334 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4ea40146a0 con 0x7f4eb406d300 2023-08-24T10:08:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.900+0000 7f4eb9aac640 1 -- 172.21.15.60:0/2988139334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb412fd30 con 0x7f4eb406d300 2023-08-24T10:08:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.900+0000 7f4e9bfff640 1 -- 172.21.15.60:0/2988139334 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4ea4014940 con 0x7f4eb406d300 2023-08-24T10:08:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.900+0000 7f4eb9aac640 1 -- 172.21.15.60:0/2988139334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb4130150 con 0x7f4eb406d300 2023-08-24T10:08:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.901+0000 7f4e9bfff640 1 -- 172.21.15.60:0/2988139334 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f4ea4014b40 con 0x7f4eb406d300 2023-08-24T10:08:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.901+0000 7f4eb9aac640 1 -- 172.21.15.60:0/2988139334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb406fe90 con 0x7f4eb406d300 2023-08-24T10:08:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.901+0000 7f4e9bfff640 1 --2- 172.21.15.60:0/2988139334 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4ea00370e0 0x7f4ea00395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:55.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.901+0000 7f4e9bfff640 1 -- 172.21.15.60:0/2988139334 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f4ea4019040 con 0x7f4eb406d300 2023-08-24T10:08:55.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.901+0000 7f4eb2ffd640 1 --2- 172.21.15.60:0/2988139334 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4ea00370e0 0x7f4ea00395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:55.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.902+0000 7f4eb2ffd640 1 --2- 172.21.15.60:0/2988139334 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4ea00370e0 0x7f4ea00395f0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:55.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:55.906+0000 7f4e9bfff640 1 -- 172.21.15.60:0/2988139334 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f4ea4018190 con 0x7f4eb406d300 2023-08-24T10:08:56.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.078+0000 7f4eb9aac640 1 -- 172.21.15.60:0/2988139334 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4eb406b170 con 0x7f4ea00370e0 2023-08-24T10:08:56.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.080+0000 7f4e9bfff640 1 -- 172.21.15.60:0/2988139334 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f4ea8001650 con 0x7f4ea00370e0 2023-08-24T10:08:56.081 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:56.082 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:56.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.082+0000 7f4eb9aac640 1 -- 172.21.15.60:0/2988139334 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4ea00370e0 msgr2=0x7f4ea00395f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:56.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.082+0000 7f4eb9aac640 1 --2- 172.21.15.60:0/2988139334 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4ea00370e0 0x7f4ea00395f0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:56.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.082+0000 7f4eb9aac640 1 -- 172.21.15.60:0/2988139334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb406d300 msgr2=0x7f4eb4096580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:56.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.082+0000 7f4eb9aac640 1 --2- 172.21.15.60:0/2988139334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb406d300 0x7f4eb4096580 crc :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:56.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.082+0000 7f4eb9aac640 1 -- 172.21.15.60:0/2988139334 shutdown_connections 2023-08-24T10:08:56.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.082+0000 7f4eb9aac640 1 --2- 172.21.15.60:0/2988139334 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4ea00370e0 0x7f4ea00395f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:56.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.082+0000 7f4eb9aac640 1 --2- 172.21.15.60:0/2988139334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb406d300 0x7f4eb4096580 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:56.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.082+0000 7f4eb9aac640 1 -- 172.21.15.60:0/2988139334 >> 172.21.15.60:0/2988139334 conn(0x7f4eb409e110 msgr2=0x7f4eb4097800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:56.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.082+0000 7f4eb9aac640 1 -- 172.21.15.60:0/2988139334 shutdown_connections 2023-08-24T10:08:56.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.082+0000 7f4eb9aac640 1 -- 172.21.15.60:0/2988139334 wait complete. 2023-08-24T10:08:56.094 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2023-08-24T10:08:55.540607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:56.096 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:56.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.204+0000 7f5f7d99e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:56.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.208+0000 7f5f7d99e640 1 Processor -- start 2023-08-24T10:08:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.208+0000 7f5f7d99e640 1 -- start start 2023-08-24T10:08:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.209+0000 7f5f7d99e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f78132b50 0x7f5f78132f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.209+0000 7f5f7d99e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f78132610 con 0x7f5f7812f110 2023-08-24T10:08:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.209+0000 7f5f7d99e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7809c260 con 0x7f5f78132b50 2023-08-24T10:08:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.209+0000 7f5f76ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f78132b50 0x7f5f78132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.209+0000 7f5f76ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f78132b50 0x7f5f78132f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33604/0 (socket says 172.21.15.60:33604) 2023-08-24T10:08:56.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.209+0000 7f5f76ffd640 1 -- 172.21.15.60:0/1933929957 learned_addr learned my addr 172.21.15.60:0/1933929957 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:56.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.210+0000 7f5f76ffd640 1 -- 172.21.15.60:0/1933929957 >> v1:172.21.15.60:6789/0 conn(0x7f5f7812f110 legacy=0x7f5f7812f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:56.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.210+0000 7f5f76ffd640 1 -- 172.21.15.60:0/1933929957 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f78132840 con 0x7f5f78132b50 2023-08-24T10:08:56.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.210+0000 7f5f76ffd640 1 --2- 172.21.15.60:0/1933929957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f78132b50 0x7f5f78132f60 crc :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98fd58dfdded4b3 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:56.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.210+0000 7f5f75ffb640 1 -- 172.21.15.60:0/1933929957 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5f6800e040 con 0x7f5f78132b50 2023-08-24T10:08:56.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.210+0000 7f5f75ffb640 1 -- 172.21.15.60:0/1933929957 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5f680091e0 con 0x7f5f78132b50 2023-08-24T10:08:56.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.210+0000 7f5f75ffb640 1 -- 172.21.15.60:0/1933929957 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5f68007730 con 0x7f5f78132b50 2023-08-24T10:08:56.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.211+0000 7f5f7d99e640 1 -- 172.21.15.60:0/1933929957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f78132b50 msgr2=0x7f5f78132f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:56.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.211+0000 7f5f7d99e640 1 --2- 172.21.15.60:0/1933929957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f78132b50 0x7f5f78132f60 crc :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:56.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.211+0000 7f5f7d99e640 1 -- 172.21.15.60:0/1933929957 shutdown_connections 2023-08-24T10:08:56.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.211+0000 7f5f7d99e640 1 --2- 172.21.15.60:0/1933929957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f78132b50 0x7f5f78132f60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:56.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.211+0000 7f5f7d99e640 1 -- 172.21.15.60:0/1933929957 >> 172.21.15.60:0/1933929957 conn(0x7f5f780a26a0 msgr2=0x7f5f780a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:56.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.211+0000 7f5f7d99e640 1 -- 172.21.15.60:0/1933929957 shutdown_connections 2023-08-24T10:08:56.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.211+0000 7f5f7d99e640 1 -- 172.21.15.60:0/1933929957 wait complete. 2023-08-24T10:08:56.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.211+0000 7f5f7d99e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:56.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.212+0000 7f5f7d99e640 1 Processor -- start 2023-08-24T10:08:56.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.212+0000 7f5f7d99e640 1 -- start start 2023-08-24T10:08:56.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.213+0000 7f5f7d99e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f7812f110 0x7f5f7809fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:56.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.213+0000 7f5f7d99e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7812cc80 con 0x7f5f7812f110 2023-08-24T10:08:56.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.213+0000 7f5f76ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f7812f110 0x7f5f7809fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:56.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.213+0000 7f5f76ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f7812f110 0x7f5f7809fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33612/0 (socket says 172.21.15.60:33612) 2023-08-24T10:08:56.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.213+0000 7f5f76ffd640 1 -- 172.21.15.60:0/990341850 learned_addr learned my addr 172.21.15.60:0/990341850 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:56.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.213+0000 7f5f76ffd640 1 -- 172.21.15.60:0/990341850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7809f400 con 0x7f5f7812f110 2023-08-24T10:08:56.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.213+0000 7f5f76ffd640 1 --2- 172.21.15.60:0/990341850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f7812f110 0x7f5f7809fcd0 crc :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:56.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.213+0000 7f5f6ffff640 1 -- 172.21.15.60:0/990341850 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5f6800e040 con 0x7f5f7812f110 2023-08-24T10:08:56.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.213+0000 7f5f6ffff640 1 -- 172.21.15.60:0/990341850 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5f68010800 con 0x7f5f7812f110 2023-08-24T10:08:56.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.213+0000 7f5f7d99e640 1 -- 172.21.15.60:0/990341850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f780a18e0 con 0x7f5f7812f110 2023-08-24T10:08:56.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.213+0000 7f5f6ffff640 1 -- 172.21.15.60:0/990341850 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5f68010aa0 con 0x7f5f7812f110 2023-08-24T10:08:56.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.213+0000 7f5f7d99e640 1 -- 172.21.15.60:0/990341850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7806a1d0 con 0x7f5f7812f110 2023-08-24T10:08:56.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.214+0000 7f5f6ffff640 1 -- 172.21.15.60:0/990341850 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f5f68010c00 con 0x7f5f7812f110 2023-08-24T10:08:56.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.214+0000 7f5f7d99e640 1 -- 172.21.15.60:0/990341850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7811c2b0 con 0x7f5f7812f110 2023-08-24T10:08:56.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.214+0000 7f5f6ffff640 1 --2- 172.21.15.60:0/990341850 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5f58037130 0x7f5f58039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:56.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.214+0000 7f5f6ffff640 1 -- 172.21.15.60:0/990341850 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f5f68047a20 con 0x7f5f7812f110 2023-08-24T10:08:56.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.215+0000 7f5f767fc640 1 --2- 172.21.15.60:0/990341850 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5f58037130 0x7f5f58039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:56.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.215+0000 7f5f767fc640 1 --2- 172.21.15.60:0/990341850 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5f58037130 0x7f5f58039640 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:56.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.219+0000 7f5f6ffff640 1 -- 172.21.15.60:0/990341850 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f5f6800e1a0 con 0x7f5f7812f110 2023-08-24T10:08:56.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.393+0000 7f5f7d99e640 1 -- 172.21.15.60:0/990341850 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f781334a0 con 0x7f5f58037130 2023-08-24T10:08:56.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.394+0000 7f5f6ffff640 1 -- 172.21.15.60:0/990341850 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f5f60001650 con 0x7f5f58037130 2023-08-24T10:08:56.396 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:56.396 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.396+0000 7f5f7d99e640 1 -- 172.21.15.60:0/990341850 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5f58037130 msgr2=0x7f5f58039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.396+0000 7f5f7d99e640 1 --2- 172.21.15.60:0/990341850 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5f58037130 0x7f5f58039640 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.396+0000 7f5f7d99e640 1 -- 172.21.15.60:0/990341850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f7812f110 msgr2=0x7f5f7809fcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.396+0000 7f5f7d99e640 1 --2- 172.21.15.60:0/990341850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f7812f110 0x7f5f7809fcd0 crc :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.396+0000 7f5f7d99e640 1 -- 172.21.15.60:0/990341850 shutdown_connections 2023-08-24T10:08:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.396+0000 7f5f7d99e640 1 --2- 172.21.15.60:0/990341850 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5f58037130 0x7f5f58039640 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.396+0000 7f5f7d99e640 1 --2- 172.21.15.60:0/990341850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f7812f110 0x7f5f7809fcd0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.396+0000 7f5f7d99e640 1 -- 172.21.15.60:0/990341850 >> 172.21.15.60:0/990341850 conn(0x7f5f780a26a0 msgr2=0x7f5f781ba920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.396+0000 7f5f7d99e640 1 -- 172.21.15.60:0/990341850 shutdown_connections 2023-08-24T10:08:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.396+0000 7f5f7d99e640 1 -- 172.21.15.60:0/990341850 wait complete. 2023-08-24T10:08:56.408 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2023-08-24T10:08:55.540607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:08:56.409 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:08:56.409 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:56.409 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:08:56.410 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:56.410 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:08:56.410 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:56.410 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:08:56.410 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:56.411 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:08:56.411 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:56.412 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:08:56.412 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:56.412 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:08:56.413 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:56.413 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:08:56.414 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:56.414 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:08:56.414 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:08:56.415 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:08:56.420 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~229s 2023-08-24T10:08:56.420 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~229s 2023-08-24T10:08:56.420 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~229s 2023-08-24T10:08:56.420 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~229s 2023-08-24T10:08:56.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.522+0000 7f3f228a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:56.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.525+0000 7f3f228a3640 1 Processor -- start 2023-08-24T10:08:56.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.525+0000 7f3f228a3640 1 -- start start 2023-08-24T10:08:56.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.526+0000 7f3f228a3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3f1c131d20 0x7f3f1c134140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:56.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.526+0000 7f3f228a3640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1c09a650 con 0x7f3f1c12ece0 2023-08-24T10:08:56.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.526+0000 7f3f228a3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1c06da00 con 0x7f3f1c131d20 2023-08-24T10:08:56.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.526+0000 7f3f1b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3f1c131d20 0x7f3f1c134140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:56.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.526+0000 7f3f1b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3f1c131d20 0x7f3f1c134140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33628/0 (socket says 172.21.15.60:33628) 2023-08-24T10:08:56.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.526+0000 7f3f1b7fe640 1 -- 172.21.15.60:0/3592013644 learned_addr learned my addr 172.21.15.60:0/3592013644 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:56.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.526+0000 7f3f1bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3f1c12ece0 0x7f3f1c12f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57438/0 (socket says 172.21.15.60:57438) 2023-08-24T10:08:56.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.526+0000 7f3f1b7fe640 1 -- 172.21.15.60:0/3592013644 >> v1:172.21.15.60:6789/0 conn(0x7f3f1c12ece0 legacy=0x7f3f1c12f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:56.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.526+0000 7f3f1b7fe640 1 -- 172.21.15.60:0/3592013644 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c12ea10 con 0x7f3f1c131d20 2023-08-24T10:08:56.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.526+0000 7f3f1b7fe640 1 --2- 172.21.15.60:0/3592013644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f1c131d20 0x7f3f1c134140 crc :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a207d7a859a201f4 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:56.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.526+0000 7f3f1affd640 1 -- 172.21.15.60:0/3592013644 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3f0c009dd0 con 0x7f3f1c131d20 2023-08-24T10:08:56.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.526+0000 7f3f1affd640 1 -- 172.21.15.60:0/3592013644 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3f0c00e230 con 0x7f3f1c131d20 2023-08-24T10:08:56.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.526+0000 7f3f1affd640 1 -- 172.21.15.60:0/3592013644 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3f0c00e4d0 con 0x7f3f1c131d20 2023-08-24T10:08:56.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.527+0000 7f3f228a3640 1 -- 172.21.15.60:0/3592013644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f1c131d20 msgr2=0x7f3f1c134140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:56.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.527+0000 7f3f228a3640 1 --2- 172.21.15.60:0/3592013644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f1c131d20 0x7f3f1c134140 crc :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:56.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.527+0000 7f3f228a3640 1 -- 172.21.15.60:0/3592013644 shutdown_connections 2023-08-24T10:08:56.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.527+0000 7f3f228a3640 1 --2- 172.21.15.60:0/3592013644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f1c131d20 0x7f3f1c134140 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:56.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.527+0000 7f3f228a3640 1 -- 172.21.15.60:0/3592013644 >> 172.21.15.60:0/3592013644 conn(0x7f3f1c09e110 msgr2=0x7f3f1c09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:56.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.527+0000 7f3f228a3640 1 -- 172.21.15.60:0/3592013644 shutdown_connections 2023-08-24T10:08:56.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.527+0000 7f3f228a3640 1 -- 172.21.15.60:0/3592013644 wait complete. 2023-08-24T10:08:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.527+0000 7f3f228a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.528+0000 7f3f228a3640 1 Processor -- start 2023-08-24T10:08:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.529+0000 7f3f228a3640 1 -- start start 2023-08-24T10:08:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.529+0000 7f3f228a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f1c12ece0 0x7f3f1c06e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.529+0000 7f3f228a3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c070040 con 0x7f3f1c12ece0 2023-08-24T10:08:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.529+0000 7f3f1bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f1c12ece0 0x7f3f1c06e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:56.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.529+0000 7f3f1bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f1c12ece0 0x7f3f1c06e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33632/0 (socket says 172.21.15.60:33632) 2023-08-24T10:08:56.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.529+0000 7f3f1bfff640 1 -- 172.21.15.60:0/2727182794 learned_addr learned my addr 172.21.15.60:0/2727182794 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:56.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.530+0000 7f3f1bfff640 1 -- 172.21.15.60:0/2727182794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c06fc90 con 0x7f3f1c12ece0 2023-08-24T10:08:56.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.530+0000 7f3f1bfff640 1 --2- 172.21.15.60:0/2727182794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f1c12ece0 0x7f3f1c06e180 crc :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:56.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.530+0000 7f3f18ff9640 1 -- 172.21.15.60:0/2727182794 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3f100071e0 con 0x7f3f1c12ece0 2023-08-24T10:08:56.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.530+0000 7f3f18ff9640 1 -- 172.21.15.60:0/2727182794 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3f10007340 con 0x7f3f1c12ece0 2023-08-24T10:08:56.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.530+0000 7f3f228a3640 1 -- 172.21.15.60:0/2727182794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1c06b200 con 0x7f3f1c12ece0 2023-08-24T10:08:56.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.530+0000 7f3f18ff9640 1 -- 172.21.15.60:0/2727182794 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3f100075e0 con 0x7f3f1c12ece0 2023-08-24T10:08:56.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.530+0000 7f3f228a3640 1 -- 172.21.15.60:0/2727182794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1c088ff0 con 0x7f3f1c12ece0 2023-08-24T10:08:56.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.531+0000 7f3f18ff9640 1 -- 172.21.15.60:0/2727182794 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f3f100057e0 con 0x7f3f1c12ece0 2023-08-24T10:08:56.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.531+0000 7f3f18ff9640 1 --2- 172.21.15.60:0/2727182794 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3f080370e0 0x7f3f080395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:56.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.531+0000 7f3f228a3640 1 -- 172.21.15.60:0/2727182794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f1c071b30 con 0x7f3f1c12ece0 2023-08-24T10:08:56.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.531+0000 7f3f18ff9640 1 -- 172.21.15.60:0/2727182794 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f3f100463f0 con 0x7f3f1c12ece0 2023-08-24T10:08:56.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.531+0000 7f3f1b7fe640 1 --2- 172.21.15.60:0/2727182794 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3f080370e0 0x7f3f080395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:56.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.532+0000 7f3f1b7fe640 1 --2- 172.21.15.60:0/2727182794 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3f080370e0 0x7f3f080395f0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:56.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.536+0000 7f3f18ff9640 1 -- 172.21.15.60:0/2727182794 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f3f1000d150 con 0x7f3f1c12ece0 2023-08-24T10:08:56.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.778+0000 7f3f228a3640 1 -- 172.21.15.60:0/2727182794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3f1c061800 con 0x7f3f1c12ece0 2023-08-24T10:08:56.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.779+0000 7f3f18ff9640 1 -- 172.21.15.60:0/2727182794 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f3f1000d510 con 0x7f3f1c12ece0 2023-08-24T10:08:56.780 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:56.781 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":247,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:08:56.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.781+0000 7f3f228a3640 1 -- 172.21.15.60:0/2727182794 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3f080370e0 msgr2=0x7f3f080395f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:56.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.781+0000 7f3f228a3640 1 --2- 172.21.15.60:0/2727182794 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3f080370e0 0x7f3f080395f0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:56.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.781+0000 7f3f228a3640 1 -- 172.21.15.60:0/2727182794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f1c12ece0 msgr2=0x7f3f1c06e180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:56.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.781+0000 7f3f228a3640 1 --2- 172.21.15.60:0/2727182794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f1c12ece0 0x7f3f1c06e180 crc :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:56.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.781+0000 7f3f228a3640 1 -- 172.21.15.60:0/2727182794 shutdown_connections 2023-08-24T10:08:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.781+0000 7f3f228a3640 1 --2- 172.21.15.60:0/2727182794 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3f080370e0 0x7f3f080395f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.781+0000 7f3f228a3640 1 --2- 172.21.15.60:0/2727182794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f1c12ece0 0x7f3f1c06e180 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.781+0000 7f3f228a3640 1 -- 172.21.15.60:0/2727182794 >> 172.21.15.60:0/2727182794 conn(0x7f3f1c09e110 msgr2=0x7f3f1c09ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.781+0000 7f3f228a3640 1 -- 172.21.15.60:0/2727182794 shutdown_connections 2023-08-24T10:08:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.781+0000 7f3f228a3640 1 -- 172.21.15.60:0/2727182794 wait complete. 2023-08-24T10:08:56.794 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:08:56.795 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:08:56.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.902+0000 7f4fcf95e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:56.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.906+0000 7f4fcf95e640 1 Processor -- start 2023-08-24T10:08:56.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.906+0000 7f4fcf95e640 1 -- start start 2023-08-24T10:08:56.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.906+0000 7f4fcf95e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4fc8132630 0x7f4fc8134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:56.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.906+0000 7f4fcf95e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc806e320 con 0x7f4fc8096860 2023-08-24T10:08:56.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.906+0000 7f4fcf95e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc806db70 con 0x7f4fc8132630 2023-08-24T10:08:56.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.907+0000 7f4fccef9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4fc8132630 0x7f4fc8134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:56.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.907+0000 7f4fccef9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4fc8132630 0x7f4fc8134a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60290/0 (socket says 172.21.15.60:60290) 2023-08-24T10:08:56.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.907+0000 7f4fccef9640 1 -- 172.21.15.60:0/4088978816 learned_addr learned my addr 172.21.15.60:0/4088978816 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:56.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.907+0000 7f4fccef9640 1 -- 172.21.15.60:0/4088978816 >> v1:172.21.15.60:6789/0 conn(0x7f4fc8096860 legacy=0x7f4fc8096c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:56.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.907+0000 7f4fccef9640 1 -- 172.21.15.60:0/4088978816 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc806cef0 con 0x7f4fc8132630 2023-08-24T10:08:56.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.907+0000 7f4fccef9640 1 --2- 172.21.15.60:0/4088978816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc8132630 0x7f4fc8134a50 crc :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98216fd295e571f0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:56.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.907+0000 7f4fbffff640 1 -- 172.21.15.60:0/4088978816 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4fb8009dd0 con 0x7f4fc8132630 2023-08-24T10:08:56.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.907+0000 7f4fbffff640 1 -- 172.21.15.60:0/4088978816 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4fb800e230 con 0x7f4fc8132630 2023-08-24T10:08:56.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.907+0000 7f4fbffff640 1 -- 172.21.15.60:0/4088978816 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4fb800e4d0 con 0x7f4fc8132630 2023-08-24T10:08:56.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.908+0000 7f4fcf95e640 1 -- 172.21.15.60:0/4088978816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc8132630 msgr2=0x7f4fc8134a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:56.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.908+0000 7f4fcf95e640 1 --2- 172.21.15.60:0/4088978816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc8132630 0x7f4fc8134a50 crc :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:56.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.908+0000 7f4fcf95e640 1 -- 172.21.15.60:0/4088978816 shutdown_connections 2023-08-24T10:08:56.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.908+0000 7f4fcf95e640 1 --2- 172.21.15.60:0/4088978816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc8132630 0x7f4fc8134a50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:56.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.908+0000 7f4fcf95e640 1 -- 172.21.15.60:0/4088978816 >> 172.21.15.60:0/4088978816 conn(0x7f4fc809e110 msgr2=0x7f4fc809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:56.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.908+0000 7f4fcf95e640 1 -- 172.21.15.60:0/4088978816 shutdown_connections 2023-08-24T10:08:56.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.908+0000 7f4fcf95e640 1 -- 172.21.15.60:0/4088978816 wait complete. 2023-08-24T10:08:56.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.908+0000 7f4fcf95e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:08:56.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.909+0000 7f4fcf95e640 1 Processor -- start 2023-08-24T10:08:56.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.909+0000 7f4fcf95e640 1 -- start start 2023-08-24T10:08:56.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.910+0000 7f4fcf95e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc8096860 0x7f4fc8127b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:56.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.910+0000 7f4fcf95e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc8129530 con 0x7f4fc8096860 2023-08-24T10:08:56.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.910+0000 7f4fcd6fa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc8096860 0x7f4fc8127b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:56.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.910+0000 7f4fcd6fa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc8096860 0x7f4fc8127b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60302/0 (socket says 172.21.15.60:60302) 2023-08-24T10:08:56.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.910+0000 7f4fcd6fa640 1 -- 172.21.15.60:0/3007952317 learned_addr learned my addr 172.21.15.60:0/3007952317 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:08:56.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.910+0000 7f4fcd6fa640 1 -- 172.21.15.60:0/3007952317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc81290c0 con 0x7f4fc8096860 2023-08-24T10:08:56.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.910+0000 7f4fcd6fa640 1 --2- 172.21.15.60:0/3007952317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc8096860 0x7f4fc8127b60 crc :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:56.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.911+0000 7f4fbdffb640 1 -- 172.21.15.60:0/3007952317 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4fc400c7e0 con 0x7f4fc8096860 2023-08-24T10:08:56.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.911+0000 7f4fbdffb640 1 -- 172.21.15.60:0/3007952317 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4fc400c940 con 0x7f4fc8096860 2023-08-24T10:08:56.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.911+0000 7f4fcf95e640 1 -- 172.21.15.60:0/3007952317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc812efa0 con 0x7f4fc8096860 2023-08-24T10:08:56.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.911+0000 7f4fbdffb640 1 -- 172.21.15.60:0/3007952317 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4fc400cbe0 con 0x7f4fc8096860 2023-08-24T10:08:56.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.911+0000 7f4fcf95e640 1 -- 172.21.15.60:0/3007952317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc8097fa0 con 0x7f4fc8096860 2023-08-24T10:08:56.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.911+0000 7f4fbdffb640 1 -- 172.21.15.60:0/3007952317 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f4fc4011310 con 0x7f4fc8096860 2023-08-24T10:08:56.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.912+0000 7f4fbdffb640 1 --2- 172.21.15.60:0/3007952317 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4fac037130 0x7f4fac039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:08:56.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.912+0000 7f4fbdffb640 1 -- 172.21.15.60:0/3007952317 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f4fc40469e0 con 0x7f4fc8096860 2023-08-24T10:08:56.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.912+0000 7f4fcf95e640 1 -- 172.21.15.60:0/3007952317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc8061800 con 0x7f4fc8096860 2023-08-24T10:08:56.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.912+0000 7f4fccef9640 1 --2- 172.21.15.60:0/3007952317 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4fac037130 0x7f4fac039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:08:56.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.913+0000 7f4fccef9640 1 --2- 172.21.15.60:0/3007952317 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4fac037130 0x7f4fac039640 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:08:56.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:56.917+0000 7f4fbdffb640 1 -- 172.21.15.60:0/3007952317 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f4fc400ad00 con 0x7f4fc8096860 2023-08-24T10:08:57.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:57.088+0000 7f4fcf95e640 1 -- 172.21.15.60:0/3007952317 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4fc806d190 con 0x7f4fac037130 2023-08-24T10:08:57.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:57.089+0000 7f4fbdffb640 1 -- 172.21.15.60:0/3007952317 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f4fb8009df0 con 0x7f4fac037130 2023-08-24T10:08:57.091 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:08:57.091 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:08:57.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:57.091+0000 7f4fcf95e640 1 -- 172.21.15.60:0/3007952317 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4fac037130 msgr2=0x7f4fac039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:57.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:57.091+0000 7f4fcf95e640 1 --2- 172.21.15.60:0/3007952317 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4fac037130 0x7f4fac039640 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:57.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:57.091+0000 7f4fcf95e640 1 -- 172.21.15.60:0/3007952317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc8096860 msgr2=0x7f4fc8127b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:08:57.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:57.091+0000 7f4fcf95e640 1 --2- 172.21.15.60:0/3007952317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc8096860 0x7f4fc8127b60 crc :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:57.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:57.091+0000 7f4fcf95e640 1 -- 172.21.15.60:0/3007952317 shutdown_connections 2023-08-24T10:08:57.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:57.091+0000 7f4fcf95e640 1 --2- 172.21.15.60:0/3007952317 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4fac037130 0x7f4fac039640 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:57.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:57.091+0000 7f4fcf95e640 1 --2- 172.21.15.60:0/3007952317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc8096860 0x7f4fc8127b60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:08:57.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:57.091+0000 7f4fcf95e640 1 -- 172.21.15.60:0/3007952317 >> 172.21.15.60:0/3007952317 conn(0x7f4fc809e110 msgr2=0x7f4fc80950c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:08:57.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:57.091+0000 7f4fcf95e640 1 -- 172.21.15.60:0/3007952317 shutdown_connections 2023-08-24T10:08:57.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:08:57.091+0000 7f4fcf95e640 1 -- 172.21.15.60:0/3007952317 wait complete. 2023-08-24T10:08:57.102 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2023-08-24T10:08:55.540607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:00.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:00.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.225+0000 7f0c35c63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:00.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.230+0000 7f0c35c63640 1 Processor -- start 2023-08-24T10:09:00.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.230+0000 7f0c35c63640 1 -- start start 2023-08-24T10:09:00.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.230+0000 7f0c35c63640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0c30131d40 0x7f0c30134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:00.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.230+0000 7f0c35c63640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c30099440 con 0x7f0c3006d300 2023-08-24T10:09:00.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.230+0000 7f0c35c63640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c3009a050 con 0x7f0c30131d40 2023-08-24T10:09:00.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.230+0000 7f0c2f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0c3006d300 0x7f0c3012bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57636/0 (socket says 172.21.15.60:57636) 2023-08-24T10:09:00.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.230+0000 7f0c2f7fe640 1 -- 172.21.15.60:0/4241587615 learned_addr learned my addr 172.21.15.60:0/4241587615 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:09:00.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.230+0000 7f0c2effd640 1 --2- 172.21.15.60:0/4241587615 >> v2:172.21.15.60:3300/0 conn(0x7f0c30131d40 0x7f0c30134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:00.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.231+0000 7f0c2effd640 1 -- 172.21.15.60:0/4241587615 >> v1:172.21.15.60:6789/0 conn(0x7f0c3006d300 legacy=0x7f0c3012bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:00.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.231+0000 7f0c2effd640 1 -- 172.21.15.60:0/4241587615 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3012b9d0 con 0x7f0c30131d40 2023-08-24T10:09:00.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.231+0000 7f0c2effd640 1 --2- 172.21.15.60:0/4241587615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c30131d40 0x7f0c30134160 crc :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da87ebbcb903d03 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:00.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.231+0000 7f0c2e7fc640 1 -- 172.21.15.60:0/4241587615 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0c24009cd0 con 0x7f0c30131d40 2023-08-24T10:09:00.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.231+0000 7f0c2e7fc640 1 -- 172.21.15.60:0/4241587615 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0c24009e30 con 0x7f0c30131d40 2023-08-24T10:09:00.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.231+0000 7f0c2e7fc640 1 -- 172.21.15.60:0/4241587615 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0c2400f470 con 0x7f0c30131d40 2023-08-24T10:09:00.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.232+0000 7f0c35c63640 1 -- 172.21.15.60:0/4241587615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c30131d40 msgr2=0x7f0c30134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:00.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.232+0000 7f0c35c63640 1 --2- 172.21.15.60:0/4241587615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c30131d40 0x7f0c30134160 crc :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:00.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.232+0000 7f0c35c63640 1 -- 172.21.15.60:0/4241587615 shutdown_connections 2023-08-24T10:09:00.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.232+0000 7f0c35c63640 1 --2- 172.21.15.60:0/4241587615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c30131d40 0x7f0c30134160 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:00.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.232+0000 7f0c35c63640 1 -- 172.21.15.60:0/4241587615 >> 172.21.15.60:0/4241587615 conn(0x7f0c3009e110 msgr2=0x7f0c3009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:00.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.232+0000 7f0c35c63640 1 -- 172.21.15.60:0/4241587615 shutdown_connections 2023-08-24T10:09:00.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.232+0000 7f0c35c63640 1 -- 172.21.15.60:0/4241587615 wait complete. 2023-08-24T10:09:00.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.232+0000 7f0c35c63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:00.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.233+0000 7f0c35c63640 1 Processor -- start 2023-08-24T10:09:00.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.233+0000 7f0c35c63640 1 -- start start 2023-08-24T10:09:00.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.234+0000 7f0c35c63640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c3006d300 0x7f0c300968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:00.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.234+0000 7f0c35c63640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c30071410 con 0x7f0c3006d300 2023-08-24T10:09:00.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.234+0000 7f0c2f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c3006d300 0x7f0c300968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:00.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.234+0000 7f0c2f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c3006d300 0x7f0c300968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60324/0 (socket says 172.21.15.60:60324) 2023-08-24T10:09:00.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.234+0000 7f0c2f7fe640 1 -- 172.21.15.60:0/774924628 learned_addr learned my addr 172.21.15.60:0/774924628 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:00.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.234+0000 7f0c2f7fe640 1 -- 172.21.15.60:0/774924628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3006f480 con 0x7f0c3006d300 2023-08-24T10:09:00.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.235+0000 7f0c2f7fe640 1 --2- 172.21.15.60:0/774924628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c3006d300 0x7f0c300968d0 crc :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:00.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.235+0000 7f0c17fff640 1 -- 172.21.15.60:0/774924628 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0c200059d0 con 0x7f0c3006d300 2023-08-24T10:09:00.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.235+0000 7f0c17fff640 1 -- 172.21.15.60:0/774924628 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0c20005b30 con 0x7f0c3006d300 2023-08-24T10:09:00.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.235+0000 7f0c35c63640 1 -- 172.21.15.60:0/774924628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c30071810 con 0x7f0c3006d300 2023-08-24T10:09:00.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.235+0000 7f0c17fff640 1 -- 172.21.15.60:0/774924628 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0c20005dd0 con 0x7f0c3006d300 2023-08-24T10:09:00.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.235+0000 7f0c35c63640 1 -- 172.21.15.60:0/774924628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3006e2d0 con 0x7f0c3006d300 2023-08-24T10:09:00.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.236+0000 7f0c17fff640 1 -- 172.21.15.60:0/774924628 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f0c200070b0 con 0x7f0c3006d300 2023-08-24T10:09:00.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.236+0000 7f0c35c63640 1 -- 172.21.15.60:0/774924628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c3006d9f0 con 0x7f0c3006d300 2023-08-24T10:09:00.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.236+0000 7f0c17fff640 1 --2- 172.21.15.60:0/774924628 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0c1c037130 0x7f0c1c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:00.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.236+0000 7f0c17fff640 1 -- 172.21.15.60:0/774924628 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f0c200462f0 con 0x7f0c3006d300 2023-08-24T10:09:00.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.236+0000 7f0c2effd640 1 --2- 172.21.15.60:0/774924628 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0c1c037130 0x7f0c1c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:00.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.237+0000 7f0c2effd640 1 --2- 172.21.15.60:0/774924628 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0c1c037130 0x7f0c1c039640 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:00.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.241+0000 7f0c17fff640 1 -- 172.21.15.60:0/774924628 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f0c2000d330 con 0x7f0c3006d300 2023-08-24T10:09:00.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.413+0000 7f0c35c63640 1 -- 172.21.15.60:0/774924628 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c3012a310 con 0x7f0c1c037130 2023-08-24T10:09:00.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.415+0000 7f0c17fff640 1 -- 172.21.15.60:0/774924628 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f0c24009c20 con 0x7f0c1c037130 2023-08-24T10:09:00.417 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:00.417 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:00.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.417+0000 7f0c35c63640 1 -- 172.21.15.60:0/774924628 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0c1c037130 msgr2=0x7f0c1c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:00.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.417+0000 7f0c35c63640 1 --2- 172.21.15.60:0/774924628 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0c1c037130 0x7f0c1c039640 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:00.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.417+0000 7f0c35c63640 1 -- 172.21.15.60:0/774924628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c3006d300 msgr2=0x7f0c300968d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:00.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.417+0000 7f0c35c63640 1 --2- 172.21.15.60:0/774924628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c3006d300 0x7f0c300968d0 crc :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:00.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.417+0000 7f0c35c63640 1 -- 172.21.15.60:0/774924628 shutdown_connections 2023-08-24T10:09:00.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.417+0000 7f0c35c63640 1 --2- 172.21.15.60:0/774924628 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0c1c037130 0x7f0c1c039640 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:00.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.417+0000 7f0c35c63640 1 --2- 172.21.15.60:0/774924628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c3006d300 0x7f0c300968d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:00.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.417+0000 7f0c35c63640 1 -- 172.21.15.60:0/774924628 >> 172.21.15.60:0/774924628 conn(0x7f0c3009e110 msgr2=0x7f0c3012ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:00.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.417+0000 7f0c35c63640 1 -- 172.21.15.60:0/774924628 shutdown_connections 2023-08-24T10:09:00.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.417+0000 7f0c35c63640 1 -- 172.21.15.60:0/774924628 wait complete. 2023-08-24T10:09:00.429 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2023-08-24T10:08:59.541400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:00.431 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:00.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.538+0000 7f46bbb9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:00.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.541+0000 7f46bbb9e640 1 Processor -- start 2023-08-24T10:09:00.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.542+0000 7f46bbb9e640 1 -- start start 2023-08-24T10:09:00.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.542+0000 7f46bbb9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f46b406d300 0x7f46b412bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:00.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.542+0000 7f46bbb9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b4099440 con 0x7f46b412c1c0 2023-08-24T10:09:00.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.542+0000 7f46bbb9e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b409a050 con 0x7f46b406d300 2023-08-24T10:09:00.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.542+0000 7f46b993a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f46b406d300 0x7f46b412bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:00.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.542+0000 7f46b993a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f46b406d300 0x7f46b412bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60338/0 (socket says 172.21.15.60:60338) 2023-08-24T10:09:00.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.542+0000 7f46b993a640 1 -- 172.21.15.60:0/2326209144 learned_addr learned my addr 172.21.15.60:0/2326209144 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:00.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.543+0000 7f46b993a640 1 -- 172.21.15.60:0/2326209144 >> v1:172.21.15.60:6789/0 conn(0x7f46b412c1c0 legacy=0x7f46b412ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:00.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.543+0000 7f46b993a640 1 -- 172.21.15.60:0/2326209144 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b412b9d0 con 0x7f46b406d300 2023-08-24T10:09:00.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.543+0000 7f46b993a640 1 --2- 172.21.15.60:0/2326209144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46b406d300 0x7f46b412bc80 crc :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560cbb9663ad9bde server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:00.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.543+0000 7f46b8938640 1 -- 172.21.15.60:0/2326209144 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f46a4009dd0 con 0x7f46b406d300 2023-08-24T10:09:00.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.543+0000 7f46b8938640 1 -- 172.21.15.60:0/2326209144 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f46a400e230 con 0x7f46b406d300 2023-08-24T10:09:00.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.543+0000 7f46b8938640 1 -- 172.21.15.60:0/2326209144 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f46a400e4d0 con 0x7f46b406d300 2023-08-24T10:09:00.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.544+0000 7f46bbb9e640 1 -- 172.21.15.60:0/2326209144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46b406d300 msgr2=0x7f46b412bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:00.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.544+0000 7f46bbb9e640 1 --2- 172.21.15.60:0/2326209144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46b406d300 0x7f46b412bc80 crc :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.544+0000 7f46bbb9e640 1 -- 172.21.15.60:0/2326209144 shutdown_connections 2023-08-24T10:09:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.544+0000 7f46bbb9e640 1 --2- 172.21.15.60:0/2326209144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46b406d300 0x7f46b412bc80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.544+0000 7f46bbb9e640 1 -- 172.21.15.60:0/2326209144 >> 172.21.15.60:0/2326209144 conn(0x7f46b409e110 msgr2=0x7f46b409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.544+0000 7f46bbb9e640 1 -- 172.21.15.60:0/2326209144 shutdown_connections 2023-08-24T10:09:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.544+0000 7f46bbb9e640 1 -- 172.21.15.60:0/2326209144 wait complete. 2023-08-24T10:09:00.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.544+0000 7f46bbb9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.545+0000 7f46bbb9e640 1 Processor -- start 2023-08-24T10:09:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.545+0000 7f46bbb9e640 1 -- start start 2023-08-24T10:09:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.546+0000 7f46bbb9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46b406d300 0x7f46b41c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.546+0000 7f46bbb9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b41c51b0 con 0x7f46b406d300 2023-08-24T10:09:00.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.546+0000 7f46b993a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46b406d300 0x7f46b41c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:00.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.546+0000 7f46b993a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46b406d300 0x7f46b41c5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60350/0 (socket says 172.21.15.60:60350) 2023-08-24T10:09:00.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.546+0000 7f46b993a640 1 -- 172.21.15.60:0/2649869843 learned_addr learned my addr 172.21.15.60:0/2649869843 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.546+0000 7f46b993a640 1 -- 172.21.15.60:0/2649869843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b4070c60 con 0x7f46b406d300 2023-08-24T10:09:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.546+0000 7f46b993a640 1 --2- 172.21.15.60:0/2649869843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46b406d300 0x7f46b41c5570 crc :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.546+0000 7f46b27fc640 1 -- 172.21.15.60:0/2649869843 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f46a4014410 con 0x7f46b406d300 2023-08-24T10:09:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.546+0000 7f46b27fc640 1 -- 172.21.15.60:0/2649869843 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f46a4014570 con 0x7f46b406d300 2023-08-24T10:09:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.546+0000 7f46bbb9e640 1 -- 172.21.15.60:0/2649869843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b4071460 con 0x7f46b406d300 2023-08-24T10:09:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.546+0000 7f46b27fc640 1 -- 172.21.15.60:0/2649869843 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f46a4014810 con 0x7f46b406d300 2023-08-24T10:09:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.546+0000 7f46bbb9e640 1 -- 172.21.15.60:0/2649869843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b41313e0 con 0x7f46b406d300 2023-08-24T10:09:00.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.547+0000 7f46bbb9e640 1 -- 172.21.15.60:0/2649869843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b406d9f0 con 0x7f46b406d300 2023-08-24T10:09:00.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.547+0000 7f46b27fc640 1 -- 172.21.15.60:0/2649869843 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f46a40149c0 con 0x7f46b406d300 2023-08-24T10:09:00.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.548+0000 7f46b27fc640 1 --2- 172.21.15.60:0/2649869843 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f46a0037130 0x7f46a0039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:00.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.548+0000 7f46b27fc640 1 -- 172.21.15.60:0/2649869843 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f46a4019040 con 0x7f46b406d300 2023-08-24T10:09:00.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.548+0000 7f46b9139640 1 --2- 172.21.15.60:0/2649869843 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f46a0037130 0x7f46a0039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:00.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.548+0000 7f46b9139640 1 --2- 172.21.15.60:0/2649869843 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f46a0037130 0x7f46a0039640 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:00.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.552+0000 7f46b27fc640 1 -- 172.21.15.60:0/2649869843 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f46a400f8f0 con 0x7f46b406d300 2023-08-24T10:09:00.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.725+0000 7f46bbb9e640 1 -- 172.21.15.60:0/2649869843 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f46b4132a70 con 0x7f46a0037130 2023-08-24T10:09:00.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.726+0000 7f46b27fc640 1 -- 172.21.15.60:0/2649869843 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f46a8001650 con 0x7f46a0037130 2023-08-24T10:09:00.728 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.728+0000 7f46bbb9e640 1 -- 172.21.15.60:0/2649869843 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f46a0037130 msgr2=0x7f46a0039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.728+0000 7f46bbb9e640 1 --2- 172.21.15.60:0/2649869843 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f46a0037130 0x7f46a0039640 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.728+0000 7f46bbb9e640 1 -- 172.21.15.60:0/2649869843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46b406d300 msgr2=0x7f46b41c5570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.728+0000 7f46bbb9e640 1 --2- 172.21.15.60:0/2649869843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46b406d300 0x7f46b41c5570 crc :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.728+0000 7f46bbb9e640 1 -- 172.21.15.60:0/2649869843 shutdown_connections 2023-08-24T10:09:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.728+0000 7f46bbb9e640 1 --2- 172.21.15.60:0/2649869843 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f46a0037130 0x7f46a0039640 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.728+0000 7f46bbb9e640 1 --2- 172.21.15.60:0/2649869843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46b406d300 0x7f46b41c5570 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.728+0000 7f46bbb9e640 1 -- 172.21.15.60:0/2649869843 >> 172.21.15.60:0/2649869843 conn(0x7f46b409e110 msgr2=0x7f46b41252d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.728+0000 7f46bbb9e640 1 -- 172.21.15.60:0/2649869843 shutdown_connections 2023-08-24T10:09:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.728+0000 7f46bbb9e640 1 -- 172.21.15.60:0/2649869843 wait complete. 2023-08-24T10:09:00.739 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2023-08-24T10:08:59.541400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:00.740 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:09:00.741 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:00.741 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:09:00.741 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:00.741 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:09:00.742 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:00.742 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:09:00.742 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:00.742 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:09:00.743 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:00.743 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:09:00.743 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:00.743 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:09:00.743 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:00.744 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:09:00.744 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:00.744 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:09:00.744 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:00.745 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:09:00.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.852+0000 7fc05d51e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:00.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.855+0000 7fc05d51e640 1 Processor -- start 2023-08-24T10:09:00.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.855+0000 7fc05d51e640 1 -- start start 2023-08-24T10:09:00.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.855+0000 7fc05d51e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc05812ece0 0x7fc05812f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:00.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.855+0000 7fc05d51e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc05809a650 con 0x7fc05812f630 2023-08-24T10:09:00.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.855+0000 7fc05d51e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc058098e30 con 0x7fc05812ece0 2023-08-24T10:09:00.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.856+0000 7fc056ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc05812ece0 0x7fc05812f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:00.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.856+0000 7fc056ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc05812ece0 0x7fc05812f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60366/0 (socket says 172.21.15.60:60366) 2023-08-24T10:09:00.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.856+0000 7fc056ffd640 1 -- 172.21.15.60:0/2094740672 learned_addr learned my addr 172.21.15.60:0/2094740672 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:00.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.856+0000 7fc04ffff640 1 --1- 172.21.15.60:0/2094740672 >> v1:172.21.15.60:6789/0 conn(0x7fc05812f630 0x7fc058097710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57656/0 (socket says 172.21.15.60:57656) 2023-08-24T10:09:00.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.856+0000 7fc056ffd640 1 -- 172.21.15.60:0/2094740672 >> v1:172.21.15.60:6789/0 conn(0x7fc05812f630 legacy=0x7fc058097710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:00.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.856+0000 7fc056ffd640 1 -- 172.21.15.60:0/2094740672 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05806da00 con 0x7fc05812ece0 2023-08-24T10:09:00.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.856+0000 7fc056ffd640 1 --2- 172.21.15.60:0/2094740672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc05812ece0 0x7fc05812f0f0 crc :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f538fcab0e64e2 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:00.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.856+0000 7fc0567fc640 1 -- 172.21.15.60:0/2094740672 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc048009dd0 con 0x7fc05812ece0 2023-08-24T10:09:00.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.856+0000 7fc0567fc640 1 -- 172.21.15.60:0/2094740672 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc04800e230 con 0x7fc05812ece0 2023-08-24T10:09:00.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.856+0000 7fc0567fc640 1 -- 172.21.15.60:0/2094740672 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc04800e4d0 con 0x7fc05812ece0 2023-08-24T10:09:00.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.857+0000 7fc05d51e640 1 -- 172.21.15.60:0/2094740672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc05812ece0 msgr2=0x7fc05812f0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:00.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.857+0000 7fc05d51e640 1 --2- 172.21.15.60:0/2094740672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc05812ece0 0x7fc05812f0f0 crc :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:00.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.857+0000 7fc05d51e640 1 -- 172.21.15.60:0/2094740672 shutdown_connections 2023-08-24T10:09:00.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.857+0000 7fc05d51e640 1 --2- 172.21.15.60:0/2094740672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc05812ece0 0x7fc05812f0f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:00.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.857+0000 7fc05d51e640 1 -- 172.21.15.60:0/2094740672 >> 172.21.15.60:0/2094740672 conn(0x7fc05809e110 msgr2=0x7fc05809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:00.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.857+0000 7fc05d51e640 1 -- 172.21.15.60:0/2094740672 shutdown_connections 2023-08-24T10:09:00.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.857+0000 7fc05d51e640 1 -- 172.21.15.60:0/2094740672 wait complete. 2023-08-24T10:09:00.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.857+0000 7fc05d51e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:00.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.858+0000 7fc05d51e640 1 Processor -- start 2023-08-24T10:09:00.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.858+0000 7fc05d51e640 1 -- start start 2023-08-24T10:09:00.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.859+0000 7fc05d51e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc05812ece0 0x7fc05812a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:00.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.859+0000 7fc05d51e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0581300c0 con 0x7fc05812ece0 2023-08-24T10:09:00.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.859+0000 7fc056ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc05812ece0 0x7fc05812a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.859+0000 7fc056ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc05812ece0 0x7fc05812a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60378/0 (socket says 172.21.15.60:60378) 2023-08-24T10:09:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.859+0000 7fc056ffd640 1 -- 172.21.15.60:0/3968482102 learned_addr learned my addr 172.21.15.60:0/3968482102 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.859+0000 7fc056ffd640 1 -- 172.21.15.60:0/3968482102 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0581305a0 con 0x7fc05812ece0 2023-08-24T10:09:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.859+0000 7fc056ffd640 1 --2- 172.21.15.60:0/3968482102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc05812ece0 0x7fc05812a8f0 crc :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.859+0000 7fc04f7fe640 1 -- 172.21.15.60:0/3968482102 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc048014540 con 0x7fc05812ece0 2023-08-24T10:09:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.859+0000 7fc04f7fe640 1 -- 172.21.15.60:0/3968482102 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc0480146a0 con 0x7fc05812ece0 2023-08-24T10:09:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.859+0000 7fc05d51e640 1 -- 172.21.15.60:0/3968482102 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05812fdb0 con 0x7fc05812ece0 2023-08-24T10:09:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.859+0000 7fc04f7fe640 1 -- 172.21.15.60:0/3968482102 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc048014940 con 0x7fc05812ece0 2023-08-24T10:09:00.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.859+0000 7fc05d51e640 1 -- 172.21.15.60:0/3968482102 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc058089f10 con 0x7fc05812ece0 2023-08-24T10:09:00.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.860+0000 7fc04f7fe640 1 -- 172.21.15.60:0/3968482102 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fc048014af0 con 0x7fc05812ece0 2023-08-24T10:09:00.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.860+0000 7fc05d51e640 1 -- 172.21.15.60:0/3968482102 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc058061800 con 0x7fc05812ece0 2023-08-24T10:09:00.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.860+0000 7fc04f7fe640 1 --2- 172.21.15.60:0/3968482102 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc038037130 0x7fc038039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:00.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.860+0000 7fc04f7fe640 1 -- 172.21.15.60:0/3968482102 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fc048019040 con 0x7fc05812ece0 2023-08-24T10:09:00.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.861+0000 7fc04ffff640 1 --2- 172.21.15.60:0/3968482102 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc038037130 0x7fc038039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:00.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.861+0000 7fc04ffff640 1 --2- 172.21.15.60:0/3968482102 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc038037130 0x7fc038039640 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:00.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:00.865+0000 7fc04f7fe640 1 -- 172.21.15.60:0/3968482102 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fc048018190 con 0x7fc05812ece0 2023-08-24T10:09:01.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.106+0000 7fc05d51e640 1 -- 172.21.15.60:0/3968482102 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc058073730 con 0x7fc05812ece0 2023-08-24T10:09:01.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.106+0000 7fc04f7fe640 1 -- 172.21.15.60:0/3968482102 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7fc048005960 con 0x7fc05812ece0 2023-08-24T10:09:01.108 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:01.108 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":251,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:09:01.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.108+0000 7fc05d51e640 1 -- 172.21.15.60:0/3968482102 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc038037130 msgr2=0x7fc038039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:01.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.108+0000 7fc05d51e640 1 --2- 172.21.15.60:0/3968482102 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc038037130 0x7fc038039640 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:01.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.108+0000 7fc05d51e640 1 -- 172.21.15.60:0/3968482102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc05812ece0 msgr2=0x7fc05812a8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:01.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.108+0000 7fc05d51e640 1 --2- 172.21.15.60:0/3968482102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc05812ece0 0x7fc05812a8f0 crc :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:01.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.108+0000 7fc05d51e640 1 -- 172.21.15.60:0/3968482102 shutdown_connections 2023-08-24T10:09:01.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.108+0000 7fc05d51e640 1 --2- 172.21.15.60:0/3968482102 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc038037130 0x7fc038039640 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:01.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.108+0000 7fc05d51e640 1 --2- 172.21.15.60:0/3968482102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc05812ece0 0x7fc05812a8f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:01.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.108+0000 7fc05d51e640 1 -- 172.21.15.60:0/3968482102 >> 172.21.15.60:0/3968482102 conn(0x7fc05809e110 msgr2=0x7fc05809c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:01.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.108+0000 7fc05d51e640 1 -- 172.21.15.60:0/3968482102 shutdown_connections 2023-08-24T10:09:01.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.108+0000 7fc05d51e640 1 -- 172.21.15.60:0/3968482102 wait complete. 2023-08-24T10:09:01.120 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:09:01.120 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:01.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.226+0000 7f7db8e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:01.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.230+0000 7f7db8e6c640 1 Processor -- start 2023-08-24T10:09:01.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.231+0000 7f7db8e6c640 1 -- start start 2023-08-24T10:09:01.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.231+0000 7f7db8e6c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7db406d300 0x7f7db412bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:01.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.231+0000 7f7db8e6c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db4099440 con 0x7f7db412c1c0 2023-08-24T10:09:01.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.231+0000 7f7db8e6c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db409a050 con 0x7f7db406d300 2023-08-24T10:09:01.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.231+0000 7f7db259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7db406d300 0x7f7db412bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:01.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.231+0000 7f7db259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7db406d300 0x7f7db412bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60388/0 (socket says 172.21.15.60:60388) 2023-08-24T10:09:01.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.231+0000 7f7db259c640 1 -- 172.21.15.60:0/2083357132 learned_addr learned my addr 172.21.15.60:0/2083357132 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:01.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.231+0000 7f7dabfff640 1 --1- 172.21.15.60:0/2083357132 >> v1:172.21.15.60:6789/0 conn(0x7f7db412c1c0 0x7f7db412ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57660/0 (socket says 172.21.15.60:57660) 2023-08-24T10:09:01.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.232+0000 7f7db259c640 1 -- 172.21.15.60:0/2083357132 >> v1:172.21.15.60:6789/0 conn(0x7f7db412c1c0 legacy=0x7f7db412ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:01.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.232+0000 7f7db259c640 1 -- 172.21.15.60:0/2083357132 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db412b9d0 con 0x7f7db406d300 2023-08-24T10:09:01.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.232+0000 7f7db259c640 1 --2- 172.21.15.60:0/2083357132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7db406d300 0x7f7db412bc80 crc :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4128994e9e6095 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:01.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.232+0000 7f7db1d9b640 1 -- 172.21.15.60:0/2083357132 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7da4009dd0 con 0x7f7db406d300 2023-08-24T10:09:01.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.232+0000 7f7db1d9b640 1 -- 172.21.15.60:0/2083357132 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7da400e230 con 0x7f7db406d300 2023-08-24T10:09:01.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.232+0000 7f7db1d9b640 1 -- 172.21.15.60:0/2083357132 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7da400e4d0 con 0x7f7db406d300 2023-08-24T10:09:01.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.233+0000 7f7db8e6c640 1 -- 172.21.15.60:0/2083357132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7db406d300 msgr2=0x7f7db412bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:01.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.233+0000 7f7db8e6c640 1 --2- 172.21.15.60:0/2083357132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7db406d300 0x7f7db412bc80 crc :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:01.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.233+0000 7f7db8e6c640 1 -- 172.21.15.60:0/2083357132 shutdown_connections 2023-08-24T10:09:01.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.233+0000 7f7db8e6c640 1 --2- 172.21.15.60:0/2083357132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7db406d300 0x7f7db412bc80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:01.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.233+0000 7f7db8e6c640 1 -- 172.21.15.60:0/2083357132 >> 172.21.15.60:0/2083357132 conn(0x7f7db409e110 msgr2=0x7f7db409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:01.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.233+0000 7f7db8e6c640 1 -- 172.21.15.60:0/2083357132 shutdown_connections 2023-08-24T10:09:01.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.233+0000 7f7db8e6c640 1 -- 172.21.15.60:0/2083357132 wait complete. 2023-08-24T10:09:01.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.233+0000 7f7db8e6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:01.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.234+0000 7f7db8e6c640 1 Processor -- start 2023-08-24T10:09:01.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.234+0000 7f7db8e6c640 1 -- start start 2023-08-24T10:09:01.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.235+0000 7f7db8e6c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7db406d300 0x7f7db4096610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:01.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.235+0000 7f7db8e6c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db4130540 con 0x7f7db406d300 2023-08-24T10:09:01.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.235+0000 7f7db259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7db406d300 0x7f7db4096610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:01.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.235+0000 7f7db259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7db406d300 0x7f7db4096610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60402/0 (socket says 172.21.15.60:60402) 2023-08-24T10:09:01.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.235+0000 7f7db259c640 1 -- 172.21.15.60:0/941775603 learned_addr learned my addr 172.21.15.60:0/941775603 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:01.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.235+0000 7f7db259c640 1 -- 172.21.15.60:0/941775603 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db4070840 con 0x7f7db406d300 2023-08-24T10:09:01.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.235+0000 7f7db259c640 1 --2- 172.21.15.60:0/941775603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7db406d300 0x7f7db4096610 crc :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:01.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.235+0000 7f7daaffd640 1 -- 172.21.15.60:0/941775603 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7da4014410 con 0x7f7db406d300 2023-08-24T10:09:01.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.235+0000 7f7daaffd640 1 -- 172.21.15.60:0/941775603 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7da4014570 con 0x7f7db406d300 2023-08-24T10:09:01.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.235+0000 7f7db8e6c640 1 -- 172.21.15.60:0/941775603 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db4135a00 con 0x7f7db406d300 2023-08-24T10:09:01.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.235+0000 7f7daaffd640 1 -- 172.21.15.60:0/941775603 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7da4014810 con 0x7f7db406d300 2023-08-24T10:09:01.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.235+0000 7f7db8e6c640 1 -- 172.21.15.60:0/941775603 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db4071860 con 0x7f7db406d300 2023-08-24T10:09:01.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.236+0000 7f7daaffd640 1 -- 172.21.15.60:0/941775603 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f7da40149c0 con 0x7f7db406d300 2023-08-24T10:09:01.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.236+0000 7f7daaffd640 1 --2- 172.21.15.60:0/941775603 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7d94037130 0x7f7d94039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:01.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.236+0000 7f7db8e6c640 1 -- 172.21.15.60:0/941775603 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db406fef0 con 0x7f7db406d300 2023-08-24T10:09:01.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.237+0000 7f7daaffd640 1 -- 172.21.15.60:0/941775603 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f7da4019040 con 0x7f7db406d300 2023-08-24T10:09:01.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.237+0000 7f7dabfff640 1 --2- 172.21.15.60:0/941775603 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7d94037130 0x7f7d94039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:01.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.237+0000 7f7dabfff640 1 --2- 172.21.15.60:0/941775603 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7d94037130 0x7f7d94039640 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:01.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.241+0000 7f7daaffd640 1 -- 172.21.15.60:0/941775603 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f7da400fb20 con 0x7f7db406d300 2023-08-24T10:09:01.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.414+0000 7f7db8e6c640 1 -- 172.21.15.60:0/941775603 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7db41303d0 con 0x7f7d94037130 2023-08-24T10:09:01.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.415+0000 7f7daaffd640 1 -- 172.21.15.60:0/941775603 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f7db41303d0 con 0x7f7d94037130 2023-08-24T10:09:01.417 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:01.417 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:01.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.417+0000 7f7db8e6c640 1 -- 172.21.15.60:0/941775603 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7d94037130 msgr2=0x7f7d94039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:01.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.417+0000 7f7db8e6c640 1 --2- 172.21.15.60:0/941775603 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7d94037130 0x7f7d94039640 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:01.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.417+0000 7f7db8e6c640 1 -- 172.21.15.60:0/941775603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7db406d300 msgr2=0x7f7db4096610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:01.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.417+0000 7f7db8e6c640 1 --2- 172.21.15.60:0/941775603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7db406d300 0x7f7db4096610 crc :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:01.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.417+0000 7f7db8e6c640 1 -- 172.21.15.60:0/941775603 shutdown_connections 2023-08-24T10:09:01.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.417+0000 7f7db8e6c640 1 --2- 172.21.15.60:0/941775603 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7d94037130 0x7f7d94039640 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:01.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.417+0000 7f7db8e6c640 1 --2- 172.21.15.60:0/941775603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7db406d300 0x7f7db4096610 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:01.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.417+0000 7f7db8e6c640 1 -- 172.21.15.60:0/941775603 >> 172.21.15.60:0/941775603 conn(0x7f7db409e110 msgr2=0x7f7db40978f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:01.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.417+0000 7f7db8e6c640 1 -- 172.21.15.60:0/941775603 shutdown_connections 2023-08-24T10:09:01.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:01.417+0000 7f7db8e6c640 1 -- 172.21.15.60:0/941775603 wait complete. 2023-08-24T10:09:01.429 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2023-08-24T10:08:59.541400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:01.623 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~234s 2023-08-24T10:09:01.623 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~234s 2023-08-24T10:09:01.623 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~234s 2023-08-24T10:09:01.624 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~234s 2023-08-24T10:09:04.430 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:04.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.551+0000 7f903295e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:04.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.555+0000 7f903295e640 1 Processor -- start 2023-08-24T10:09:04.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.555+0000 7f903295e640 1 -- start start 2023-08-24T10:09:04.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.556+0000 7f903295e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f902c132b50 0x7f902c132f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:04.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.556+0000 7f903295e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f902c132610 con 0x7f902c12f110 2023-08-24T10:09:04.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.556+0000 7f903295e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f902c09c260 con 0x7f902c132b50 2023-08-24T10:09:04.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.556+0000 7f902bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f902c132b50 0x7f902c132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:04.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.556+0000 7f902bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f902c132b50 0x7f902c132f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60414/0 (socket says 172.21.15.60:60414) 2023-08-24T10:09:04.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.556+0000 7f902b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f902c12f110 0x7f902c12f520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57662/0 (socket says 172.21.15.60:57662) 2023-08-24T10:09:04.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.556+0000 7f902bfff640 1 -- 172.21.15.60:0/294020081 learned_addr learned my addr 172.21.15.60:0/294020081 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:04.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.557+0000 7f902bfff640 1 -- 172.21.15.60:0/294020081 >> v1:172.21.15.60:6789/0 conn(0x7f902c12f110 legacy=0x7f902c12f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:04.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.557+0000 7f902bfff640 1 -- 172.21.15.60:0/294020081 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c132840 con 0x7f902c132b50 2023-08-24T10:09:04.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.557+0000 7f902bfff640 1 --2- 172.21.15.60:0/294020081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f902c132b50 0x7f902c132f60 crc :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698937cf393ac355 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:04.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.557+0000 7f902affd640 1 -- 172.21.15.60:0/294020081 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f901c009dd0 con 0x7f902c132b50 2023-08-24T10:09:04.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.557+0000 7f902affd640 1 -- 172.21.15.60:0/294020081 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f901c00e230 con 0x7f902c132b50 2023-08-24T10:09:04.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.557+0000 7f902affd640 1 -- 172.21.15.60:0/294020081 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f901c00e4d0 con 0x7f902c132b50 2023-08-24T10:09:04.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.558+0000 7f903295e640 1 -- 172.21.15.60:0/294020081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f902c132b50 msgr2=0x7f902c132f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:04.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.558+0000 7f903295e640 1 --2- 172.21.15.60:0/294020081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f902c132b50 0x7f902c132f60 crc :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:04.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.558+0000 7f903295e640 1 -- 172.21.15.60:0/294020081 shutdown_connections 2023-08-24T10:09:04.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.558+0000 7f903295e640 1 --2- 172.21.15.60:0/294020081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f902c132b50 0x7f902c132f60 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:04.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.558+0000 7f903295e640 1 -- 172.21.15.60:0/294020081 >> 172.21.15.60:0/294020081 conn(0x7f902c0a26a0 msgr2=0x7f902c0a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:04.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.558+0000 7f903295e640 1 -- 172.21.15.60:0/294020081 shutdown_connections 2023-08-24T10:09:04.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.558+0000 7f903295e640 1 -- 172.21.15.60:0/294020081 wait complete. 2023-08-24T10:09:04.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.558+0000 7f903295e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:04.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.559+0000 7f903295e640 1 Processor -- start 2023-08-24T10:09:04.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.559+0000 7f903295e640 1 -- start start 2023-08-24T10:09:04.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.560+0000 7f903295e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f902c12f110 0x7f902c09fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:04.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.560+0000 7f903295e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902c12cc80 con 0x7f902c12f110 2023-08-24T10:09:04.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.560+0000 7f902bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f902c12f110 0x7f902c09fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:04.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.560+0000 7f902bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f902c12f110 0x7f902c09fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60422/0 (socket says 172.21.15.60:60422) 2023-08-24T10:09:04.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.560+0000 7f902bfff640 1 -- 172.21.15.60:0/505446512 learned_addr learned my addr 172.21.15.60:0/505446512 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:04.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.560+0000 7f902bfff640 1 -- 172.21.15.60:0/505446512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c09f400 con 0x7f902c12f110 2023-08-24T10:09:04.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.560+0000 7f902bfff640 1 --2- 172.21.15.60:0/505446512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f902c12f110 0x7f902c09fcd0 crc :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:04.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.560+0000 7f9028ff9640 1 -- 172.21.15.60:0/505446512 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f901c014410 con 0x7f902c12f110 2023-08-24T10:09:04.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.560+0000 7f9028ff9640 1 -- 172.21.15.60:0/505446512 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f901c014570 con 0x7f902c12f110 2023-08-24T10:09:04.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.560+0000 7f903295e640 1 -- 172.21.15.60:0/505446512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f902c0a18e0 con 0x7f902c12f110 2023-08-24T10:09:04.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.560+0000 7f9028ff9640 1 -- 172.21.15.60:0/505446512 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f901c014810 con 0x7f902c12f110 2023-08-24T10:09:04.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.561+0000 7f903295e640 1 -- 172.21.15.60:0/505446512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f902c06a1d0 con 0x7f902c12f110 2023-08-24T10:09:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.562+0000 7f9028ff9640 1 -- 172.21.15.60:0/505446512 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f901c0149c0 con 0x7f902c12f110 2023-08-24T10:09:04.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.562+0000 7f903295e640 1 -- 172.21.15.60:0/505446512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f902c11c2b0 con 0x7f902c12f110 2023-08-24T10:09:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.562+0000 7f9028ff9640 1 --2- 172.21.15.60:0/505446512 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f900c037130 0x7f900c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.562+0000 7f9028ff9640 1 -- 172.21.15.60:0/505446512 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f901c019040 con 0x7f902c12f110 2023-08-24T10:09:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.562+0000 7f902b7fe640 1 --2- 172.21.15.60:0/505446512 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f900c037130 0x7f900c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.563+0000 7f902b7fe640 1 --2- 172.21.15.60:0/505446512 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f900c037130 0x7f900c039640 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:04.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.567+0000 7f9028ff9640 1 -- 172.21.15.60:0/505446512 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f901c00f8f0 con 0x7f902c12f110 2023-08-24T10:09:04.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.741+0000 7f903295e640 1 -- 172.21.15.60:0/505446512 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f902c1334a0 con 0x7f900c037130 2023-08-24T10:09:04.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.742+0000 7f9028ff9640 1 -- 172.21.15.60:0/505446512 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f90140070b0 con 0x7f900c037130 2023-08-24T10:09:04.744 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:04.744 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:04.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.744+0000 7f903295e640 1 -- 172.21.15.60:0/505446512 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f900c037130 msgr2=0x7f900c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.744+0000 7f903295e640 1 --2- 172.21.15.60:0/505446512 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f900c037130 0x7f900c039640 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.744+0000 7f903295e640 1 -- 172.21.15.60:0/505446512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f902c12f110 msgr2=0x7f902c09fcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.744+0000 7f903295e640 1 --2- 172.21.15.60:0/505446512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f902c12f110 0x7f902c09fcd0 crc :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.744+0000 7f903295e640 1 -- 172.21.15.60:0/505446512 shutdown_connections 2023-08-24T10:09:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.744+0000 7f903295e640 1 --2- 172.21.15.60:0/505446512 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f900c037130 0x7f900c039640 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.744+0000 7f903295e640 1 --2- 172.21.15.60:0/505446512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f902c12f110 0x7f902c09fcd0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.744+0000 7f903295e640 1 -- 172.21.15.60:0/505446512 >> 172.21.15.60:0/505446512 conn(0x7f902c0a26a0 msgr2=0x7f902c1ba920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:04.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.745+0000 7f903295e640 1 -- 172.21.15.60:0/505446512 shutdown_connections 2023-08-24T10:09:04.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.745+0000 7f903295e640 1 -- 172.21.15.60:0/505446512 wait complete. 2023-08-24T10:09:04.756 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2023-08-24T10:09:03.542227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:04.757 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:04.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.863+0000 7f506caec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:04.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.867+0000 7f506caec640 1 Processor -- start 2023-08-24T10:09:04.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.867+0000 7f506caec640 1 -- start start 2023-08-24T10:09:04.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.868+0000 7f506caec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f506806d300 0x7f5068097710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:04.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.868+0000 7f506caec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5068099440 con 0x7f506812bc80 2023-08-24T10:09:04.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.868+0000 7f506caec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f506809a050 con 0x7f506806d300 2023-08-24T10:09:04.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.868+0000 7f506659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f506806d300 0x7f5068097710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:04.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.868+0000 7f506659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f506806d300 0x7f5068097710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60438/0 (socket says 172.21.15.60:60438) 2023-08-24T10:09:04.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.868+0000 7f506659c640 1 -- 172.21.15.60:0/1064757470 learned_addr learned my addr 172.21.15.60:0/1064757470 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:04.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.868+0000 7f506659c640 1 -- 172.21.15.60:0/1064757470 >> v1:172.21.15.60:6789/0 conn(0x7f506812bc80 legacy=0x7f506812ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:04.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.868+0000 7f506659c640 1 -- 172.21.15.60:0/1064757470 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506812b9d0 con 0x7f506806d300 2023-08-24T10:09:04.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.869+0000 7f506659c640 1 --2- 172.21.15.60:0/1064757470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f506806d300 0x7f5068097710 crc :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991a9330db0307c1 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:04.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.869+0000 7f506559a640 1 -- 172.21.15.60:0/1064757470 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5058009dc0 con 0x7f506806d300 2023-08-24T10:09:04.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.869+0000 7f506559a640 1 -- 172.21.15.60:0/1064757470 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f505800e230 con 0x7f506806d300 2023-08-24T10:09:04.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.869+0000 7f506559a640 1 -- 172.21.15.60:0/1064757470 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f505800e4d0 con 0x7f506806d300 2023-08-24T10:09:04.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.869+0000 7f506caec640 1 -- 172.21.15.60:0/1064757470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f506806d300 msgr2=0x7f5068097710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:04.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.869+0000 7f506caec640 1 --2- 172.21.15.60:0/1064757470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f506806d300 0x7f5068097710 crc :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:04.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.869+0000 7f506caec640 1 -- 172.21.15.60:0/1064757470 shutdown_connections 2023-08-24T10:09:04.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.869+0000 7f506caec640 1 --2- 172.21.15.60:0/1064757470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f506806d300 0x7f5068097710 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:04.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.869+0000 7f506caec640 1 -- 172.21.15.60:0/1064757470 >> 172.21.15.60:0/1064757470 conn(0x7f506809e110 msgr2=0x7f506809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:04.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.870+0000 7f506caec640 1 -- 172.21.15.60:0/1064757470 shutdown_connections 2023-08-24T10:09:04.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.870+0000 7f506caec640 1 -- 172.21.15.60:0/1064757470 wait complete. 2023-08-24T10:09:04.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.870+0000 7f506caec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:04.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.871+0000 7f506caec640 1 Processor -- start 2023-08-24T10:09:04.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.871+0000 7f506caec640 1 -- start start 2023-08-24T10:09:04.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.871+0000 7f506caec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f506806d300 0x7f5068070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:04.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.871+0000 7f506caec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5068089870 con 0x7f506806d300 2023-08-24T10:09:04.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.872+0000 7f506659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f506806d300 0x7f5068070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:04.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.872+0000 7f506659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f506806d300 0x7f5068070210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60446/0 (socket says 172.21.15.60:60446) 2023-08-24T10:09:04.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.872+0000 7f506659c640 1 -- 172.21.15.60:0/1862426415 learned_addr learned my addr 172.21.15.60:0/1862426415 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:04.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.872+0000 7f506659c640 1 -- 172.21.15.60:0/1862426415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506806b0e0 con 0x7f506806d300 2023-08-24T10:09:04.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.872+0000 7f506659c640 1 --2- 172.21.15.60:0/1862426415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f506806d300 0x7f5068070210 crc :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:04.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.872+0000 7f5056ffd640 1 -- 172.21.15.60:0/1862426415 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5058014540 con 0x7f506806d300 2023-08-24T10:09:04.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.872+0000 7f5056ffd640 1 -- 172.21.15.60:0/1862426415 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f50580146a0 con 0x7f506806d300 2023-08-24T10:09:04.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.872+0000 7f506caec640 1 -- 172.21.15.60:0/1862426415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5068071660 con 0x7f506806d300 2023-08-24T10:09:04.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.872+0000 7f5056ffd640 1 -- 172.21.15.60:0/1862426415 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5058014940 con 0x7f506806d300 2023-08-24T10:09:04.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.872+0000 7f506caec640 1 -- 172.21.15.60:0/1862426415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5068088ff0 con 0x7f506806d300 2023-08-24T10:09:04.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.873+0000 7f5056ffd640 1 -- 172.21.15.60:0/1862426415 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f5058014af0 con 0x7f506806d300 2023-08-24T10:09:04.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.873+0000 7f5056ffd640 1 --2- 172.21.15.60:0/1862426415 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5048037130 0x7f5048039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:04.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.873+0000 7f5056ffd640 1 -- 172.21.15.60:0/1862426415 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f5058019040 con 0x7f506806d300 2023-08-24T10:09:04.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.873+0000 7f506caec640 1 -- 172.21.15.60:0/1862426415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f506808a390 con 0x7f506806d300 2023-08-24T10:09:04.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.873+0000 7f5065d9b640 1 --2- 172.21.15.60:0/1862426415 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5048037130 0x7f5048039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:04.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.874+0000 7f5065d9b640 1 --2- 172.21.15.60:0/1862426415 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5048037130 0x7f5048039640 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:04.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:04.878+0000 7f5056ffd640 1 -- 172.21.15.60:0/1862426415 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f5058014df0 con 0x7f506806d300 2023-08-24T10:09:05.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.051+0000 7f506caec640 1 -- 172.21.15.60:0/1862426415 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f506806d710 con 0x7f5048037130 2023-08-24T10:09:05.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.052+0000 7f5056ffd640 1 -- 172.21.15.60:0/1862426415 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f505c001650 con 0x7f5048037130 2023-08-24T10:09:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:05.054 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:05.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.054+0000 7f506caec640 1 -- 172.21.15.60:0/1862426415 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5048037130 msgr2=0x7f5048039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:05.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.054+0000 7f506caec640 1 --2- 172.21.15.60:0/1862426415 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5048037130 0x7f5048039640 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:05.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.054+0000 7f506caec640 1 -- 172.21.15.60:0/1862426415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f506806d300 msgr2=0x7f5068070210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:05.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.054+0000 7f506caec640 1 --2- 172.21.15.60:0/1862426415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f506806d300 0x7f5068070210 crc :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:05.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.054+0000 7f506caec640 1 -- 172.21.15.60:0/1862426415 shutdown_connections 2023-08-24T10:09:05.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.054+0000 7f506caec640 1 --2- 172.21.15.60:0/1862426415 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5048037130 0x7f5048039640 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:05.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.054+0000 7f506caec640 1 --2- 172.21.15.60:0/1862426415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f506806d300 0x7f5068070210 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:05.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.054+0000 7f506caec640 1 -- 172.21.15.60:0/1862426415 >> 172.21.15.60:0/1862426415 conn(0x7f506809e110 msgr2=0x7f506809afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:05.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.054+0000 7f506caec640 1 -- 172.21.15.60:0/1862426415 shutdown_connections 2023-08-24T10:09:05.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.054+0000 7f506caec640 1 -- 172.21.15.60:0/1862426415 wait complete. 2023-08-24T10:09:05.066 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2023-08-24T10:09:03.542227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:05.067 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:09:05.067 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:05.068 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:09:05.068 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:05.068 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:09:05.068 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:05.069 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:09:05.069 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:05.069 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:09:05.069 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:05.070 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:09:05.070 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:05.071 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:09:05.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:05.071 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:09:05.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:05.072 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:09:05.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:05.073 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:09:05.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.180+0000 7efcc695e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.183+0000 7efcc695e640 1 Processor -- start 2023-08-24T10:09:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.183+0000 7efcc695e640 1 -- start start 2023-08-24T10:09:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.184+0000 7efcc695e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efcc0133700 0x7efcc0135b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.184+0000 7efcc695e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc009a1e0 con 0x7efcc0096410 2023-08-24T10:09:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.184+0000 7efcc695e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc0097af0 con 0x7efcc0133700 2023-08-24T10:09:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.184+0000 7efcbf7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efcc0133700 0x7efcc0135b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.184+0000 7efcbf7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efcc0133700 0x7efcc0135b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60462/0 (socket says 172.21.15.60:60462) 2023-08-24T10:09:05.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.184+0000 7efcbf7fe640 1 -- 172.21.15.60:0/1487481364 learned_addr learned my addr 172.21.15.60:0/1487481364 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:05.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.184+0000 7efcbffff640 1 --1- 172.21.15.60:0/1487481364 >> v1:172.21.15.60:6789/0 conn(0x7efcc0096410 0x7efcc0096820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57682/0 (socket says 172.21.15.60:57682) 2023-08-24T10:09:05.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.184+0000 7efcbf7fe640 1 -- 172.21.15.60:0/1487481364 >> v1:172.21.15.60:6789/0 conn(0x7efcc0096410 legacy=0x7efcc0096820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:05.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.184+0000 7efcbf7fe640 1 -- 172.21.15.60:0/1487481364 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc0096f30 con 0x7efcc0133700 2023-08-24T10:09:05.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.184+0000 7efcbf7fe640 1 --2- 172.21.15.60:0/1487481364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc0133700 0x7efcc0135b20 crc :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4391e48e302206f4 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:05.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.185+0000 7efcbeffd640 1 -- 172.21.15.60:0/1487481364 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efcb0009dd0 con 0x7efcc0133700 2023-08-24T10:09:05.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.185+0000 7efcbeffd640 1 -- 172.21.15.60:0/1487481364 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7efcb000e230 con 0x7efcc0133700 2023-08-24T10:09:05.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.185+0000 7efcbeffd640 1 -- 172.21.15.60:0/1487481364 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efcb000e4d0 con 0x7efcc0133700 2023-08-24T10:09:05.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.185+0000 7efcc695e640 1 -- 172.21.15.60:0/1487481364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc0133700 msgr2=0x7efcc0135b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:05.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.185+0000 7efcc695e640 1 --2- 172.21.15.60:0/1487481364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc0133700 0x7efcc0135b20 crc :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:05.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.185+0000 7efcc695e640 1 -- 172.21.15.60:0/1487481364 shutdown_connections 2023-08-24T10:09:05.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.185+0000 7efcc695e640 1 --2- 172.21.15.60:0/1487481364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc0133700 0x7efcc0135b20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:05.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.185+0000 7efcc695e640 1 -- 172.21.15.60:0/1487481364 >> 172.21.15.60:0/1487481364 conn(0x7efcc009e110 msgr2=0x7efcc009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:05.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.185+0000 7efcc695e640 1 -- 172.21.15.60:0/1487481364 shutdown_connections 2023-08-24T10:09:05.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.185+0000 7efcc695e640 1 -- 172.21.15.60:0/1487481364 wait complete. 2023-08-24T10:09:05.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.186+0000 7efcc695e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:05.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.186+0000 7efcc695e640 1 Processor -- start 2023-08-24T10:09:05.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.187+0000 7efcc695e640 1 -- start start 2023-08-24T10:09:05.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.187+0000 7efcc695e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc0096410 0x7efcc012f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:05.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.187+0000 7efcc695e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc006d6f0 con 0x7efcc0096410 2023-08-24T10:09:05.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.188+0000 7efcbffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc0096410 0x7efcc012f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:05.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.188+0000 7efcbffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc0096410 0x7efcc012f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60476/0 (socket says 172.21.15.60:60476) 2023-08-24T10:09:05.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.188+0000 7efcbffff640 1 -- 172.21.15.60:0/1082073201 learned_addr learned my addr 172.21.15.60:0/1082073201 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:05.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.188+0000 7efcbffff640 1 -- 172.21.15.60:0/1082073201 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc006ef90 con 0x7efcc0096410 2023-08-24T10:09:05.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.188+0000 7efcbffff640 1 --2- 172.21.15.60:0/1082073201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc0096410 0x7efcc012f380 crc :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:05.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.188+0000 7efcbcff9640 1 -- 172.21.15.60:0/1082073201 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efcb40071e0 con 0x7efcc0096410 2023-08-24T10:09:05.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.188+0000 7efcbcff9640 1 -- 172.21.15.60:0/1082073201 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7efcb4007340 con 0x7efcc0096410 2023-08-24T10:09:05.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.188+0000 7efcc695e640 1 -- 172.21.15.60:0/1082073201 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc006d9e0 con 0x7efcc0096410 2023-08-24T10:09:05.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.188+0000 7efcbcff9640 1 -- 172.21.15.60:0/1082073201 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efcb40075e0 con 0x7efcc0096410 2023-08-24T10:09:05.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.188+0000 7efcc695e640 1 -- 172.21.15.60:0/1082073201 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc006d230 con 0x7efcc0096410 2023-08-24T10:09:05.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.189+0000 7efcbcff9640 1 -- 172.21.15.60:0/1082073201 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7efcb40057e0 con 0x7efcc0096410 2023-08-24T10:09:05.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.189+0000 7efcc695e640 1 -- 172.21.15.60:0/1082073201 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc009cb20 con 0x7efcc0096410 2023-08-24T10:09:05.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.189+0000 7efcbcff9640 1 --2- 172.21.15.60:0/1082073201 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efcac037130 0x7efcac039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:05.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.189+0000 7efcbcff9640 1 -- 172.21.15.60:0/1082073201 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7efcb4046470 con 0x7efcc0096410 2023-08-24T10:09:05.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.190+0000 7efcbf7fe640 1 --2- 172.21.15.60:0/1082073201 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efcac037130 0x7efcac039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:05.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.190+0000 7efcbf7fe640 1 --2- 172.21.15.60:0/1082073201 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efcac037130 0x7efcac039640 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:05.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.194+0000 7efcbcff9640 1 -- 172.21.15.60:0/1082073201 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7efcc009cb20 con 0x7efcc0096410 2023-08-24T10:09:05.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.440+0000 7efcc695e640 1 -- 172.21.15.60:0/1082073201 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7efcc009cb20 con 0x7efcc0096410 2023-08-24T10:09:05.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.441+0000 7efcbcff9640 1 -- 172.21.15.60:0/1082073201 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7efcc009cb20 con 0x7efcc0096410 2023-08-24T10:09:05.442 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:05.442 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":256,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:09:05.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.443+0000 7efcc695e640 1 -- 172.21.15.60:0/1082073201 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efcac037130 msgr2=0x7efcac039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:05.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.443+0000 7efcc695e640 1 --2- 172.21.15.60:0/1082073201 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efcac037130 0x7efcac039640 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:05.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.443+0000 7efcc695e640 1 -- 172.21.15.60:0/1082073201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc0096410 msgr2=0x7efcc012f380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:05.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.443+0000 7efcc695e640 1 --2- 172.21.15.60:0/1082073201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc0096410 0x7efcc012f380 crc :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:05.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.443+0000 7efcc695e640 1 -- 172.21.15.60:0/1082073201 shutdown_connections 2023-08-24T10:09:05.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.443+0000 7efcc695e640 1 --2- 172.21.15.60:0/1082073201 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7efcac037130 0x7efcac039640 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:05.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.443+0000 7efcc695e640 1 --2- 172.21.15.60:0/1082073201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcc0096410 0x7efcc012f380 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:05.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.443+0000 7efcc695e640 1 -- 172.21.15.60:0/1082073201 >> 172.21.15.60:0/1082073201 conn(0x7efcc009e110 msgr2=0x7efcc00985b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:05.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.443+0000 7efcc695e640 1 -- 172.21.15.60:0/1082073201 shutdown_connections 2023-08-24T10:09:05.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.443+0000 7efcc695e640 1 -- 172.21.15.60:0/1082073201 wait complete. 2023-08-24T10:09:05.454 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:09:05.455 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:05.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.561+0000 7f14508ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:05.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.565+0000 7f14508ec640 1 Processor -- start 2023-08-24T10:09:05.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.565+0000 7f14508ec640 1 -- start start 2023-08-24T10:09:05.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.566+0000 7f14508ec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f144c06d300 0x7f144c097710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:05.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.566+0000 7f14508ec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f144c099440 con 0x7f144c12bc80 2023-08-24T10:09:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.566+0000 7f14508ec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f144c09a050 con 0x7f144c06d300 2023-08-24T10:09:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.566+0000 7f144a59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f144c06d300 0x7f144c097710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.566+0000 7f144a59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f144c06d300 0x7f144c097710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60482/0 (socket says 172.21.15.60:60482) 2023-08-24T10:09:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.566+0000 7f144a59c640 1 -- 172.21.15.60:0/1410869923 learned_addr learned my addr 172.21.15.60:0/1410869923 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.566+0000 7f1449d9b640 1 --1- 172.21.15.60:0/1410869923 >> v1:172.21.15.60:6789/0 conn(0x7f144c12bc80 0x7f144c12ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57684/0 (socket says 172.21.15.60:57684) 2023-08-24T10:09:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.566+0000 7f144a59c640 1 -- 172.21.15.60:0/1410869923 >> v1:172.21.15.60:6789/0 conn(0x7f144c12bc80 legacy=0x7f144c12ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.566+0000 7f144a59c640 1 -- 172.21.15.60:0/1410869923 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144c12b9d0 con 0x7f144c06d300 2023-08-24T10:09:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.566+0000 7f144a59c640 1 --2- 172.21.15.60:0/1410869923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c06d300 0x7f144c097710 crc :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78734ba9d136671 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.567+0000 7f144959a640 1 -- 172.21.15.60:0/1410869923 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f143c00e040 con 0x7f144c06d300 2023-08-24T10:09:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.567+0000 7f144959a640 1 -- 172.21.15.60:0/1410869923 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f143c0091e0 con 0x7f144c06d300 2023-08-24T10:09:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.567+0000 7f144959a640 1 -- 172.21.15.60:0/1410869923 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f143c007730 con 0x7f144c06d300 2023-08-24T10:09:05.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.567+0000 7f14508ec640 1 -- 172.21.15.60:0/1410869923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c06d300 msgr2=0x7f144c097710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:05.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.567+0000 7f14508ec640 1 --2- 172.21.15.60:0/1410869923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c06d300 0x7f144c097710 crc :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:05.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.567+0000 7f14508ec640 1 -- 172.21.15.60:0/1410869923 shutdown_connections 2023-08-24T10:09:05.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.567+0000 7f14508ec640 1 --2- 172.21.15.60:0/1410869923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c06d300 0x7f144c097710 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:05.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.567+0000 7f14508ec640 1 -- 172.21.15.60:0/1410869923 >> 172.21.15.60:0/1410869923 conn(0x7f144c09e110 msgr2=0x7f144c09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:05.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.567+0000 7f14508ec640 1 -- 172.21.15.60:0/1410869923 shutdown_connections 2023-08-24T10:09:05.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.567+0000 7f14508ec640 1 -- 172.21.15.60:0/1410869923 wait complete. 2023-08-24T10:09:05.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.568+0000 7f14508ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:05.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.569+0000 7f14508ec640 1 Processor -- start 2023-08-24T10:09:05.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.569+0000 7f14508ec640 1 -- start start 2023-08-24T10:09:05.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.569+0000 7f14508ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c06d300 0x7f144c06e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:05.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.569+0000 7f14508ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c06e4e0 con 0x7f144c06d300 2023-08-24T10:09:05.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.569+0000 7f144a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c06d300 0x7f144c06e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:05.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.569+0000 7f144a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c06d300 0x7f144c06e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60484/0 (socket says 172.21.15.60:60484) 2023-08-24T10:09:05.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.569+0000 7f144a59c640 1 -- 172.21.15.60:0/451110696 learned_addr learned my addr 172.21.15.60:0/451110696 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:05.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.570+0000 7f144a59c640 1 -- 172.21.15.60:0/451110696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144c070530 con 0x7f144c06d300 2023-08-24T10:09:05.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.570+0000 7f144a59c640 1 --2- 172.21.15.60:0/451110696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c06d300 0x7f144c06e700 crc :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:05.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.570+0000 7f143affd640 1 -- 172.21.15.60:0/451110696 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f143c00e040 con 0x7f144c06d300 2023-08-24T10:09:05.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.570+0000 7f143affd640 1 -- 172.21.15.60:0/451110696 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f143c010790 con 0x7f144c06d300 2023-08-24T10:09:05.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.570+0000 7f14508ec640 1 -- 172.21.15.60:0/451110696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144c096ef0 con 0x7f144c06d300 2023-08-24T10:09:05.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.570+0000 7f143affd640 1 -- 172.21.15.60:0/451110696 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f143c010a30 con 0x7f144c06d300 2023-08-24T10:09:05.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.570+0000 7f14508ec640 1 -- 172.21.15.60:0/451110696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144c08d5f0 con 0x7f144c06d300 2023-08-24T10:09:05.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.571+0000 7f143affd640 1 -- 172.21.15.60:0/451110696 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f143c010b90 con 0x7f144c06d300 2023-08-24T10:09:05.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.571+0000 7f143affd640 1 --2- 172.21.15.60:0/451110696 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f142c037130 0x7f142c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:05.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.571+0000 7f14508ec640 1 -- 172.21.15.60:0/451110696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f144c06fc50 con 0x7f144c06d300 2023-08-24T10:09:05.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.571+0000 7f143affd640 1 -- 172.21.15.60:0/451110696 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f143c0479f0 con 0x7f144c06d300 2023-08-24T10:09:05.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.571+0000 7f1449d9b640 1 --2- 172.21.15.60:0/451110696 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f142c037130 0x7f142c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:05.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.572+0000 7f1449d9b640 1 --2- 172.21.15.60:0/451110696 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f142c037130 0x7f142c039640 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:05.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.576+0000 7f143affd640 1 -- 172.21.15.60:0/451110696 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f143c0143e0 con 0x7f144c06d300 2023-08-24T10:09:05.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.746+0000 7f14508ec640 1 -- 172.21.15.60:0/451110696 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f144c06da20 con 0x7f142c037130 2023-08-24T10:09:05.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.747+0000 7f143affd640 1 -- 172.21.15.60:0/451110696 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f1440007e80 con 0x7f142c037130 2023-08-24T10:09:05.749 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.749+0000 7f14508ec640 1 -- 172.21.15.60:0/451110696 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f142c037130 msgr2=0x7f142c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.749+0000 7f14508ec640 1 --2- 172.21.15.60:0/451110696 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f142c037130 0x7f142c039640 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.749+0000 7f14508ec640 1 -- 172.21.15.60:0/451110696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c06d300 msgr2=0x7f144c06e700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.749+0000 7f14508ec640 1 --2- 172.21.15.60:0/451110696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c06d300 0x7f144c06e700 crc :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.749+0000 7f14508ec640 1 -- 172.21.15.60:0/451110696 shutdown_connections 2023-08-24T10:09:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.749+0000 7f14508ec640 1 --2- 172.21.15.60:0/451110696 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f142c037130 0x7f142c039640 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.749+0000 7f14508ec640 1 --2- 172.21.15.60:0/451110696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f144c06d300 0x7f144c06e700 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.749+0000 7f14508ec640 1 -- 172.21.15.60:0/451110696 >> 172.21.15.60:0/451110696 conn(0x7f144c09e110 msgr2=0x7f144c09afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.749+0000 7f14508ec640 1 -- 172.21.15.60:0/451110696 shutdown_connections 2023-08-24T10:09:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:05.749+0000 7f14508ec640 1 -- 172.21.15.60:0/451110696 wait complete. 2023-08-24T10:09:05.761 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2023-08-24T10:09:05.542636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:06.827 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~239s 2023-08-24T10:09:06.827 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~239s 2023-08-24T10:09:06.827 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~239s 2023-08-24T10:09:06.828 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~239s 2023-08-24T10:09:08.763 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:08.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.883+0000 7fe821e63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:08.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.887+0000 7fe821e63640 1 Processor -- start 2023-08-24T10:09:08.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.887+0000 7fe821e63640 1 -- start start 2023-08-24T10:09:08.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.888+0000 7fe821e63640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe81c06d300 0x7fe81c12bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:08.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.888+0000 7fe821e63640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81c099440 con 0x7fe81c12c1c0 2023-08-24T10:09:08.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.888+0000 7fe821e63640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81c09a050 con 0x7fe81c06d300 2023-08-24T10:09:08.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.888+0000 7fe81b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe81c06d300 0x7fe81c12bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:08.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.888+0000 7fe81b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe81c06d300 0x7fe81c12bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35712/0 (socket says 172.21.15.60:35712) 2023-08-24T10:09:08.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.888+0000 7fe81affd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe81c12c1c0 0x7fe81c12ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34808/0 (socket says 172.21.15.60:34808) 2023-08-24T10:09:08.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.888+0000 7fe81b7fe640 1 -- 172.21.15.60:0/552400751 learned_addr learned my addr 172.21.15.60:0/552400751 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:08.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.888+0000 7fe81b7fe640 1 -- 172.21.15.60:0/552400751 >> v1:172.21.15.60:6789/0 conn(0x7fe81c12c1c0 legacy=0x7fe81c12ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:08.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.888+0000 7fe81b7fe640 1 -- 172.21.15.60:0/552400751 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c12b9d0 con 0x7fe81c06d300 2023-08-24T10:09:08.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.888+0000 7fe81b7fe640 1 --2- 172.21.15.60:0/552400751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81c06d300 0x7fe81c12bc80 crc :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ffe77ad16eead3 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:08.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.889+0000 7fe81a7fc640 1 -- 172.21.15.60:0/552400751 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe80c00d580 con 0x7fe81c06d300 2023-08-24T10:09:08.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.889+0000 7fe81a7fc640 1 -- 172.21.15.60:0/552400751 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fe80c00d6e0 con 0x7fe81c06d300 2023-08-24T10:09:08.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.889+0000 7fe81a7fc640 1 -- 172.21.15.60:0/552400751 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe80c00d980 con 0x7fe81c06d300 2023-08-24T10:09:08.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.889+0000 7fe821e63640 1 -- 172.21.15.60:0/552400751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81c06d300 msgr2=0x7fe81c12bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:08.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.889+0000 7fe821e63640 1 --2- 172.21.15.60:0/552400751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81c06d300 0x7fe81c12bc80 crc :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:08.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.889+0000 7fe821e63640 1 -- 172.21.15.60:0/552400751 shutdown_connections 2023-08-24T10:09:08.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.889+0000 7fe821e63640 1 --2- 172.21.15.60:0/552400751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81c06d300 0x7fe81c12bc80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:08.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.889+0000 7fe821e63640 1 -- 172.21.15.60:0/552400751 >> 172.21.15.60:0/552400751 conn(0x7fe81c09e110 msgr2=0x7fe81c09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:08.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.889+0000 7fe821e63640 1 -- 172.21.15.60:0/552400751 shutdown_connections 2023-08-24T10:09:08.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.889+0000 7fe821e63640 1 -- 172.21.15.60:0/552400751 wait complete. 2023-08-24T10:09:08.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.890+0000 7fe821e63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:08.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.891+0000 7fe821e63640 1 Processor -- start 2023-08-24T10:09:08.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.891+0000 7fe821e63640 1 -- start start 2023-08-24T10:09:08.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.891+0000 7fe821e63640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81c06d300 0x7fe81c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:08.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.891+0000 7fe821e63640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c089370 con 0x7fe81c06d300 2023-08-24T10:09:08.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.892+0000 7fe81b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81c06d300 0x7fe81c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:08.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.892+0000 7fe81b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81c06d300 0x7fe81c06e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35714/0 (socket says 172.21.15.60:35714) 2023-08-24T10:09:08.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.892+0000 7fe81b7fe640 1 -- 172.21.15.60:0/3521197645 learned_addr learned my addr 172.21.15.60:0/3521197645 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:08.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.892+0000 7fe81b7fe640 1 -- 172.21.15.60:0/3521197645 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c096f10 con 0x7fe81c06d300 2023-08-24T10:09:08.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.892+0000 7fe81b7fe640 1 --2- 172.21.15.60:0/3521197645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81c06d300 0x7fe81c06e9f0 crc :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:08.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.892+0000 7fe803fff640 1 -- 172.21.15.60:0/3521197645 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe80c00c9a0 con 0x7fe81c06d300 2023-08-24T10:09:08.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.892+0000 7fe803fff640 1 -- 172.21.15.60:0/3521197645 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fe80c00cb00 con 0x7fe81c06d300 2023-08-24T10:09:08.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.892+0000 7fe821e63640 1 -- 172.21.15.60:0/3521197645 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81c089560 con 0x7fe81c06d300 2023-08-24T10:09:08.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.892+0000 7fe803fff640 1 -- 172.21.15.60:0/3521197645 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe80c00cda0 con 0x7fe81c06d300 2023-08-24T10:09:08.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.892+0000 7fe821e63640 1 -- 172.21.15.60:0/3521197645 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c071780 con 0x7fe81c06d300 2023-08-24T10:09:08.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.893+0000 7fe803fff640 1 -- 172.21.15.60:0/3521197645 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fe80c017430 con 0x7fe81c06d300 2023-08-24T10:09:08.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.893+0000 7fe821e63640 1 -- 172.21.15.60:0/3521197645 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe81c06e840 con 0x7fe81c06d300 2023-08-24T10:09:08.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.894+0000 7fe803fff640 1 --2- 172.21.15.60:0/3521197645 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe808037130 0x7fe808039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:08.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.894+0000 7fe803fff640 1 -- 172.21.15.60:0/3521197645 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fe80c012040 con 0x7fe81c06d300 2023-08-24T10:09:08.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.894+0000 7fe81affd640 1 --2- 172.21.15.60:0/3521197645 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe808037130 0x7fe808039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:08.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.894+0000 7fe81affd640 1 --2- 172.21.15.60:0/3521197645 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe808037130 0x7fe808039640 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:08.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:08.898+0000 7fe803fff640 1 -- 172.21.15.60:0/3521197645 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fe80c0142e0 con 0x7fe81c06d300 2023-08-24T10:09:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.070+0000 7fe821e63640 1 -- 172.21.15.60:0/3521197645 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe81c06da20 con 0x7fe808037130 2023-08-24T10:09:09.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.072+0000 7fe803fff640 1 -- 172.21.15.60:0/3521197645 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7fe810007e80 con 0x7fe808037130 2023-08-24T10:09:09.074 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:09.075 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:09.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.074+0000 7fe821e63640 1 -- 172.21.15.60:0/3521197645 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe808037130 msgr2=0x7fe808039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:09.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.074+0000 7fe821e63640 1 --2- 172.21.15.60:0/3521197645 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe808037130 0x7fe808039640 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:09.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.074+0000 7fe821e63640 1 -- 172.21.15.60:0/3521197645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81c06d300 msgr2=0x7fe81c06e9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:09.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.074+0000 7fe821e63640 1 --2- 172.21.15.60:0/3521197645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81c06d300 0x7fe81c06e9f0 crc :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.074+0000 7fe821e63640 1 -- 172.21.15.60:0/3521197645 shutdown_connections 2023-08-24T10:09:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.074+0000 7fe821e63640 1 --2- 172.21.15.60:0/3521197645 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe808037130 0x7fe808039640 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.074+0000 7fe821e63640 1 --2- 172.21.15.60:0/3521197645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe81c06d300 0x7fe81c06e9f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.074+0000 7fe821e63640 1 -- 172.21.15.60:0/3521197645 >> 172.21.15.60:0/3521197645 conn(0x7fe81c09e110 msgr2=0x7fe81c09bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.074+0000 7fe821e63640 1 -- 172.21.15.60:0/3521197645 shutdown_connections 2023-08-24T10:09:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.074+0000 7fe821e63640 1 -- 172.21.15.60:0/3521197645 wait complete. 2023-08-24T10:09:09.085 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2023-08-24T10:09:07.543085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:09.086 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:09.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.193+0000 7f2a718ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:09.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.197+0000 7f2a718ac640 1 Processor -- start 2023-08-24T10:09:09.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.197+0000 7f2a718ac640 1 -- start start 2023-08-24T10:09:09.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.198+0000 7f2a718ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2a6c096860 0x7f2a6c096c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:09.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.198+0000 7f2a718ac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a6c0971b0 con 0x7f2a6c0990b0 2023-08-24T10:09:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.198+0000 7f2a718ac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a6c06db70 con 0x7f2a6c096860 2023-08-24T10:09:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.198+0000 7f2a6affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2a6c096860 0x7f2a6c096c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.198+0000 7f2a6affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2a6c096860 0x7f2a6c096c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35718/0 (socket says 172.21.15.60:35718) 2023-08-24T10:09:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.198+0000 7f2a6affd640 1 -- 172.21.15.60:0/3923753749 learned_addr learned my addr 172.21.15.60:0/3923753749 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.198+0000 7f2a63fff640 1 --1- 172.21.15.60:0/3923753749 >> v1:172.21.15.60:6789/0 conn(0x7f2a6c0990b0 0x7f2a6c0994c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34814/0 (socket says 172.21.15.60:34814) 2023-08-24T10:09:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.198+0000 7f2a6affd640 1 -- 172.21.15.60:0/3923753749 >> v1:172.21.15.60:6789/0 conn(0x7f2a6c0990b0 legacy=0x7f2a6c0994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.198+0000 7f2a6affd640 1 -- 172.21.15.60:0/3923753749 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6c06e320 con 0x7f2a6c096860 2023-08-24T10:09:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.198+0000 7f2a6affd640 1 --2- 172.21.15.60:0/3923753749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a6c096860 0x7f2a6c096c70 crc :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e6371a0ac000cc server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.198+0000 7f2a6a7fc640 1 -- 172.21.15.60:0/3923753749 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2a5c009dd0 con 0x7f2a6c096860 2023-08-24T10:09:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.198+0000 7f2a6a7fc640 1 -- 172.21.15.60:0/3923753749 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f2a5c00e230 con 0x7f2a6c096860 2023-08-24T10:09:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.198+0000 7f2a6a7fc640 1 -- 172.21.15.60:0/3923753749 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2a5c00e4d0 con 0x7f2a6c096860 2023-08-24T10:09:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.199+0000 7f2a718ac640 1 -- 172.21.15.60:0/3923753749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a6c096860 msgr2=0x7f2a6c096c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.199+0000 7f2a718ac640 1 --2- 172.21.15.60:0/3923753749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a6c096860 0x7f2a6c096c70 crc :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.199+0000 7f2a718ac640 1 -- 172.21.15.60:0/3923753749 shutdown_connections 2023-08-24T10:09:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.199+0000 7f2a718ac640 1 --2- 172.21.15.60:0/3923753749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a6c096860 0x7f2a6c096c70 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:09.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.199+0000 7f2a718ac640 1 -- 172.21.15.60:0/3923753749 >> 172.21.15.60:0/3923753749 conn(0x7f2a6c09e110 msgr2=0x7f2a6c09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:09.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.199+0000 7f2a718ac640 1 -- 172.21.15.60:0/3923753749 shutdown_connections 2023-08-24T10:09:09.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.199+0000 7f2a718ac640 1 -- 172.21.15.60:0/3923753749 wait complete. 2023-08-24T10:09:09.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.199+0000 7f2a718ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:09.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.200+0000 7f2a718ac640 1 Processor -- start 2023-08-24T10:09:09.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.200+0000 7f2a718ac640 1 -- start start 2023-08-24T10:09:09.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.201+0000 7f2a718ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a6c096860 0x7f2a6c12af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:09.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.201+0000 7f2a718ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c129890 con 0x7f2a6c096860 2023-08-24T10:09:09.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.201+0000 7f2a6affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a6c096860 0x7f2a6c12af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:09.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.201+0000 7f2a6affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a6c096860 0x7f2a6c12af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35732/0 (socket says 172.21.15.60:35732) 2023-08-24T10:09:09.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.201+0000 7f2a6affd640 1 -- 172.21.15.60:0/892184951 learned_addr learned my addr 172.21.15.60:0/892184951 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:09.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.201+0000 7f2a6affd640 1 -- 172.21.15.60:0/892184951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6c1bf650 con 0x7f2a6c096860 2023-08-24T10:09:09.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.201+0000 7f2a6affd640 1 --2- 172.21.15.60:0/892184951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a6c096860 0x7f2a6c12af10 crc :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:09.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.201+0000 7f2a68ff9640 1 -- 172.21.15.60:0/892184951 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2a5c014540 con 0x7f2a6c096860 2023-08-24T10:09:09.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.202+0000 7f2a68ff9640 1 -- 172.21.15.60:0/892184951 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f2a5c0146a0 con 0x7f2a6c096860 2023-08-24T10:09:09.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.202+0000 7f2a68ff9640 1 -- 172.21.15.60:0/892184951 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2a5c014940 con 0x7f2a6c096860 2023-08-24T10:09:09.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.202+0000 7f2a718ac640 1 -- 172.21.15.60:0/892184951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a6c1bf960 con 0x7f2a6c096860 2023-08-24T10:09:09.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.202+0000 7f2a718ac640 1 -- 172.21.15.60:0/892184951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a6c12a940 con 0x7f2a6c096860 2023-08-24T10:09:09.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.202+0000 7f2a68ff9640 1 -- 172.21.15.60:0/892184951 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f2a5c014af0 con 0x7f2a6c096860 2023-08-24T10:09:09.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.203+0000 7f2a718ac640 1 -- 172.21.15.60:0/892184951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a6c061800 con 0x7f2a6c096860 2023-08-24T10:09:09.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.203+0000 7f2a68ff9640 1 --2- 172.21.15.60:0/892184951 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2a4c037130 0x7f2a4c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:09.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.203+0000 7f2a68ff9640 1 -- 172.21.15.60:0/892184951 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f2a5c019040 con 0x7f2a6c096860 2023-08-24T10:09:09.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.203+0000 7f2a63fff640 1 --2- 172.21.15.60:0/892184951 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2a4c037130 0x7f2a4c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:09.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.204+0000 7f2a63fff640 1 --2- 172.21.15.60:0/892184951 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2a4c037130 0x7f2a4c039640 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:09.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.207+0000 7f2a68ff9640 1 -- 172.21.15.60:0/892184951 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f2a5c018190 con 0x7f2a6c096860 2023-08-24T10:09:09.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.382+0000 7f2a718ac640 1 -- 172.21.15.60:0/892184951 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a6c12bf20 con 0x7f2a4c037130 2023-08-24T10:09:09.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.384+0000 7f2a68ff9640 1 -- 172.21.15.60:0/892184951 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f2a6c12bf20 con 0x7f2a4c037130 2023-08-24T10:09:09.385 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:09.385 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:09.387 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.386+0000 7f2a718ac640 1 -- 172.21.15.60:0/892184951 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2a4c037130 msgr2=0x7f2a4c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:09.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.386+0000 7f2a718ac640 1 --2- 172.21.15.60:0/892184951 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2a4c037130 0x7f2a4c039640 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:09.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.386+0000 7f2a718ac640 1 -- 172.21.15.60:0/892184951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a6c096860 msgr2=0x7f2a6c12af10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:09.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.386+0000 7f2a718ac640 1 --2- 172.21.15.60:0/892184951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a6c096860 0x7f2a6c12af10 crc :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:09.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.386+0000 7f2a718ac640 1 -- 172.21.15.60:0/892184951 shutdown_connections 2023-08-24T10:09:09.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.386+0000 7f2a718ac640 1 --2- 172.21.15.60:0/892184951 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2a4c037130 0x7f2a4c039640 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:09.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.386+0000 7f2a718ac640 1 --2- 172.21.15.60:0/892184951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a6c096860 0x7f2a6c12af10 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:09.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.386+0000 7f2a718ac640 1 -- 172.21.15.60:0/892184951 >> 172.21.15.60:0/892184951 conn(0x7f2a6c09e110 msgr2=0x7f2a6c1b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:09.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.387+0000 7f2a718ac640 1 -- 172.21.15.60:0/892184951 shutdown_connections 2023-08-24T10:09:09.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.387+0000 7f2a718ac640 1 -- 172.21.15.60:0/892184951 wait complete. 2023-08-24T10:09:09.399 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2023-08-24T10:09:07.543085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:09.401 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:09:09.402 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:09.403 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:09:09.403 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:09.404 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:09:09.404 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:09.405 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:09:09.405 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:09.405 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:09:09.406 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:09.406 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:09:09.406 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:09.406 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:09:09.407 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:09.407 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:09:09.407 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:09.407 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:09:09.408 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:09.408 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:09:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.516+0000 7fc5940ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.518+0000 7fc5940ec640 1 Processor -- start 2023-08-24T10:09:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.519+0000 7fc5940ec640 1 -- start start 2023-08-24T10:09:09.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.519+0000 7fc5940ec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc58c12f610 0x7fc58c12fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:09.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.519+0000 7fc5940ec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc58c09d750 con 0x7fc58c1326f0 2023-08-24T10:09:09.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.519+0000 7fc5940ec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc58c09d980 con 0x7fc58c12f610 2023-08-24T10:09:09.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.519+0000 7fc591e88640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc58c12f610 0x7fc58c12fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:09.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.519+0000 7fc591e88640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc58c12f610 0x7fc58c12fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35744/0 (socket says 172.21.15.60:35744) 2023-08-24T10:09:09.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.520+0000 7fc591e88640 1 -- 172.21.15.60:0/3130112451 learned_addr learned my addr 172.21.15.60:0/3130112451 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:09.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.519+0000 7fc591687640 1 --1- 172.21.15.60:0/3130112451 >> v1:172.21.15.60:6789/0 conn(0x7fc58c1326f0 0x7fc58c132b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34824/0 (socket says 172.21.15.60:34824) 2023-08-24T10:09:09.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.520+0000 7fc591e88640 1 -- 172.21.15.60:0/3130112451 >> v1:172.21.15.60:6789/0 conn(0x7fc58c1326f0 legacy=0x7fc58c132b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:09.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.520+0000 7fc591e88640 1 -- 172.21.15.60:0/3130112451 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58c09bde0 con 0x7fc58c12f610 2023-08-24T10:09:09.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.520+0000 7fc591e88640 1 --2- 172.21.15.60:0/3130112451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc58c12f610 0x7fc58c12fa20 crc :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf149785c5802fed server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:09.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.520+0000 7fc590e86640 1 -- 172.21.15.60:0/3130112451 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc588009dc0 con 0x7fc58c12f610 2023-08-24T10:09:09.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.520+0000 7fc590e86640 1 -- 172.21.15.60:0/3130112451 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc58800e230 con 0x7fc58c12f610 2023-08-24T10:09:09.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.520+0000 7fc590e86640 1 -- 172.21.15.60:0/3130112451 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc58800e4d0 con 0x7fc58c12f610 2023-08-24T10:09:09.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.521+0000 7fc5940ec640 1 -- 172.21.15.60:0/3130112451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc58c12f610 msgr2=0x7fc58c12fa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:09.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.521+0000 7fc5940ec640 1 --2- 172.21.15.60:0/3130112451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc58c12f610 0x7fc58c12fa20 crc :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:09.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.521+0000 7fc5940ec640 1 -- 172.21.15.60:0/3130112451 shutdown_connections 2023-08-24T10:09:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.521+0000 7fc5940ec640 1 --2- 172.21.15.60:0/3130112451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc58c12f610 0x7fc58c12fa20 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.521+0000 7fc5940ec640 1 -- 172.21.15.60:0/3130112451 >> 172.21.15.60:0/3130112451 conn(0x7fc58c0a2500 msgr2=0x7fc58c0a2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.521+0000 7fc5940ec640 1 -- 172.21.15.60:0/3130112451 shutdown_connections 2023-08-24T10:09:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.521+0000 7fc5940ec640 1 -- 172.21.15.60:0/3130112451 wait complete. 2023-08-24T10:09:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.521+0000 7fc5940ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.522+0000 7fc5940ec640 1 Processor -- start 2023-08-24T10:09:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.522+0000 7fc5940ec640 1 -- start start 2023-08-24T10:09:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.523+0000 7fc5940ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc58c12f610 0x7fc58c09ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.523+0000 7fc5940ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c09fa60 con 0x7fc58c12f610 2023-08-24T10:09:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.523+0000 7fc591e88640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc58c12f610 0x7fc58c09ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.523+0000 7fc591e88640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc58c12f610 0x7fc58c09ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35754/0 (socket says 172.21.15.60:35754) 2023-08-24T10:09:09.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.523+0000 7fc591e88640 1 -- 172.21.15.60:0/3769484284 learned_addr learned my addr 172.21.15.60:0/3769484284 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:09.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.523+0000 7fc591e88640 1 -- 172.21.15.60:0/3769484284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58c0a1120 con 0x7fc58c12f610 2023-08-24T10:09:09.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.523+0000 7fc591e88640 1 --2- 172.21.15.60:0/3769484284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc58c12f610 0x7fc58c09ffb0 crc :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:09.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.523+0000 7fc5827fc640 1 -- 172.21.15.60:0/3769484284 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc588014540 con 0x7fc58c12f610 2023-08-24T10:09:09.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.523+0000 7fc5827fc640 1 -- 172.21.15.60:0/3769484284 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc5880146a0 con 0x7fc58c12f610 2023-08-24T10:09:09.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.523+0000 7fc5940ec640 1 -- 172.21.15.60:0/3769484284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58c11adf0 con 0x7fc58c12f610 2023-08-24T10:09:09.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.523+0000 7fc5827fc640 1 -- 172.21.15.60:0/3769484284 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc588014940 con 0x7fc58c12f610 2023-08-24T10:09:09.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.523+0000 7fc5940ec640 1 -- 172.21.15.60:0/3769484284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58c11c010 con 0x7fc58c12f610 2023-08-24T10:09:09.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.524+0000 7fc5827fc640 1 -- 172.21.15.60:0/3769484284 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fc588014af0 con 0x7fc58c12f610 2023-08-24T10:09:09.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.524+0000 7fc5827fc640 1 --2- 172.21.15.60:0/3769484284 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc570037130 0x7fc570039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:09.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.524+0000 7fc5827fc640 1 -- 172.21.15.60:0/3769484284 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fc588019040 con 0x7fc58c12f610 2023-08-24T10:09:09.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.524+0000 7fc5940ec640 1 -- 172.21.15.60:0/3769484284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc58c061800 con 0x7fc58c12f610 2023-08-24T10:09:09.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.525+0000 7fc591687640 1 --2- 172.21.15.60:0/3769484284 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc570037130 0x7fc570039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:09.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.525+0000 7fc591687640 1 --2- 172.21.15.60:0/3769484284 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc570037130 0x7fc570039640 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:09.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.529+0000 7fc5827fc640 1 -- 172.21.15.60:0/3769484284 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fc588014df0 con 0x7fc58c12f610 2023-08-24T10:09:09.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.776+0000 7fc5940ec640 1 -- 172.21.15.60:0/3769484284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc58c09d440 con 0x7fc58c12f610 2023-08-24T10:09:09.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.777+0000 7fc5827fc640 1 -- 172.21.15.60:0/3769484284 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7fc588005960 con 0x7fc58c12f610 2023-08-24T10:09:09.779 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:09.779 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":260,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:09:09.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.779+0000 7fc5940ec640 1 -- 172.21.15.60:0/3769484284 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc570037130 msgr2=0x7fc570039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:09.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.779+0000 7fc5940ec640 1 --2- 172.21.15.60:0/3769484284 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc570037130 0x7fc570039640 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:09.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.779+0000 7fc5940ec640 1 -- 172.21.15.60:0/3769484284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc58c12f610 msgr2=0x7fc58c09ffb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:09.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.779+0000 7fc5940ec640 1 --2- 172.21.15.60:0/3769484284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc58c12f610 0x7fc58c09ffb0 crc :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:09.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.779+0000 7fc5940ec640 1 -- 172.21.15.60:0/3769484284 shutdown_connections 2023-08-24T10:09:09.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.779+0000 7fc5940ec640 1 --2- 172.21.15.60:0/3769484284 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc570037130 0x7fc570039640 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:09.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.779+0000 7fc5940ec640 1 --2- 172.21.15.60:0/3769484284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc58c12f610 0x7fc58c09ffb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:09.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.779+0000 7fc5940ec640 1 -- 172.21.15.60:0/3769484284 >> 172.21.15.60:0/3769484284 conn(0x7fc58c0a2500 msgr2=0x7fc58c1ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:09.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.779+0000 7fc5940ec640 1 -- 172.21.15.60:0/3769484284 shutdown_connections 2023-08-24T10:09:09.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.779+0000 7fc5940ec640 1 -- 172.21.15.60:0/3769484284 wait complete. 2023-08-24T10:09:09.790 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:09:09.791 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:09.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.897+0000 7f4934e56640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:09.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.901+0000 7f4934e56640 1 Processor -- start 2023-08-24T10:09:09.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.901+0000 7f4934e56640 1 -- start start 2023-08-24T10:09:09.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.902+0000 7f4934e56640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4930132b50 0x7f4930132f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:09.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.902+0000 7f4934e56640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4930132610 con 0x7f493012f110 2023-08-24T10:09:09.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.902+0000 7f4934e56640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493009c260 con 0x7f4930132b50 2023-08-24T10:09:09.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.902+0000 7f492e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4930132b50 0x7f4930132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:09.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.902+0000 7f492e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4930132b50 0x7f4930132f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35766/0 (socket says 172.21.15.60:35766) 2023-08-24T10:09:09.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.902+0000 7f492e59c640 1 -- 172.21.15.60:0/2127832185 learned_addr learned my addr 172.21.15.60:0/2127832185 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:09.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.903+0000 7f492e59c640 1 -- 172.21.15.60:0/2127832185 >> v1:172.21.15.60:6789/0 conn(0x7f493012f110 legacy=0x7f493012f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:09.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.903+0000 7f492e59c640 1 -- 172.21.15.60:0/2127832185 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4930132840 con 0x7f4930132b50 2023-08-24T10:09:09.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.903+0000 7f492e59c640 1 --2- 172.21.15.60:0/2127832185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4930132b50 0x7f4930132f60 crc :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5e0d6b740e9462 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:09.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.903+0000 7f492d59a640 1 -- 172.21.15.60:0/2127832185 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4928009dd0 con 0x7f4930132b50 2023-08-24T10:09:09.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.903+0000 7f492d59a640 1 -- 172.21.15.60:0/2127832185 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f492800e230 con 0x7f4930132b50 2023-08-24T10:09:09.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.903+0000 7f492d59a640 1 -- 172.21.15.60:0/2127832185 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f492800e4d0 con 0x7f4930132b50 2023-08-24T10:09:09.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.904+0000 7f4934e56640 1 -- 172.21.15.60:0/2127832185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4930132b50 msgr2=0x7f4930132f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:09.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.904+0000 7f4934e56640 1 --2- 172.21.15.60:0/2127832185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4930132b50 0x7f4930132f60 crc :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:09.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.904+0000 7f4934e56640 1 -- 172.21.15.60:0/2127832185 shutdown_connections 2023-08-24T10:09:09.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.904+0000 7f4934e56640 1 --2- 172.21.15.60:0/2127832185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4930132b50 0x7f4930132f60 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:09.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.904+0000 7f4934e56640 1 -- 172.21.15.60:0/2127832185 >> 172.21.15.60:0/2127832185 conn(0x7f49300a26a0 msgr2=0x7f49300a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:09.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.904+0000 7f4934e56640 1 -- 172.21.15.60:0/2127832185 shutdown_connections 2023-08-24T10:09:09.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.904+0000 7f4934e56640 1 -- 172.21.15.60:0/2127832185 wait complete. 2023-08-24T10:09:09.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.904+0000 7f4934e56640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:09.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.905+0000 7f4934e56640 1 Processor -- start 2023-08-24T10:09:09.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.905+0000 7f4934e56640 1 -- start start 2023-08-24T10:09:09.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.906+0000 7f4934e56640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f493012f110 0x7f493009fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:09.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.906+0000 7f4934e56640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493012cc80 con 0x7f493012f110 2023-08-24T10:09:09.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.906+0000 7f492e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f493012f110 0x7f493009fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:09.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.906+0000 7f492e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f493012f110 0x7f493009fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35780/0 (socket says 172.21.15.60:35780) 2023-08-24T10:09:09.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.906+0000 7f492e59c640 1 -- 172.21.15.60:0/3129035260 learned_addr learned my addr 172.21.15.60:0/3129035260 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:09.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.906+0000 7f492e59c640 1 -- 172.21.15.60:0/3129035260 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493009f400 con 0x7f493012f110 2023-08-24T10:09:09.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.906+0000 7f492e59c640 1 --2- 172.21.15.60:0/3129035260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f493012f110 0x7f493009fcd0 crc :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:09.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.906+0000 7f4926ffd640 1 -- 172.21.15.60:0/3129035260 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4928014410 con 0x7f493012f110 2023-08-24T10:09:09.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.906+0000 7f4926ffd640 1 -- 172.21.15.60:0/3129035260 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4928014570 con 0x7f493012f110 2023-08-24T10:09:09.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.906+0000 7f4934e56640 1 -- 172.21.15.60:0/3129035260 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49300a18e0 con 0x7f493012f110 2023-08-24T10:09:09.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.906+0000 7f4926ffd640 1 -- 172.21.15.60:0/3129035260 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4928014810 con 0x7f493012f110 2023-08-24T10:09:09.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.906+0000 7f4934e56640 1 -- 172.21.15.60:0/3129035260 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493006a1d0 con 0x7f493012f110 2023-08-24T10:09:09.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.907+0000 7f4926ffd640 1 -- 172.21.15.60:0/3129035260 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f49280149c0 con 0x7f493012f110 2023-08-24T10:09:09.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.908+0000 7f4926ffd640 1 --2- 172.21.15.60:0/3129035260 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4910037180 0x7f4910039690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:09.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.908+0000 7f4934e56640 1 -- 172.21.15.60:0/3129035260 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493011c2b0 con 0x7f493012f110 2023-08-24T10:09:09.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.908+0000 7f4926ffd640 1 -- 172.21.15.60:0/3129035260 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f4928019040 con 0x7f493012f110 2023-08-24T10:09:09.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.908+0000 7f492dd9b640 1 --2- 172.21.15.60:0/3129035260 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4910037180 0x7f4910039690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:09.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.908+0000 7f492dd9b640 1 --2- 172.21.15.60:0/3129035260 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4910037180 0x7f4910039690 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:09.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:09.912+0000 7f4926ffd640 1 -- 172.21.15.60:0/3129035260 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f4928047e60 con 0x7f493012f110 2023-08-24T10:09:10.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:10.083+0000 7f4934e56640 1 -- 172.21.15.60:0/3129035260 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f49301334a0 con 0x7f4910037180 2023-08-24T10:09:10.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:10.085+0000 7f4926ffd640 1 -- 172.21.15.60:0/3129035260 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f4918001650 con 0x7f4910037180 2023-08-24T10:09:10.086 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:10.086 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:10.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:10.086+0000 7f4934e56640 1 -- 172.21.15.60:0/3129035260 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4910037180 msgr2=0x7f4910039690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:10.086+0000 7f4934e56640 1 --2- 172.21.15.60:0/3129035260 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4910037180 0x7f4910039690 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:10.086+0000 7f4934e56640 1 -- 172.21.15.60:0/3129035260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f493012f110 msgr2=0x7f493009fcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:10.086+0000 7f4934e56640 1 --2- 172.21.15.60:0/3129035260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f493012f110 0x7f493009fcd0 crc :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:10.086+0000 7f4934e56640 1 -- 172.21.15.60:0/3129035260 shutdown_connections 2023-08-24T10:09:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:10.086+0000 7f4934e56640 1 --2- 172.21.15.60:0/3129035260 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4910037180 0x7f4910039690 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:10.086+0000 7f4934e56640 1 --2- 172.21.15.60:0/3129035260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f493012f110 0x7f493009fcd0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:10.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:10.086+0000 7f4934e56640 1 -- 172.21.15.60:0/3129035260 >> 172.21.15.60:0/3129035260 conn(0x7f49300a26a0 msgr2=0x7f49301ba920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:10.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:10.086+0000 7f4934e56640 1 -- 172.21.15.60:0/3129035260 shutdown_connections 2023-08-24T10:09:10.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:10.087+0000 7f4934e56640 1 -- 172.21.15.60:0/3129035260 wait complete. 2023-08-24T10:09:10.098 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2023-08-24T10:09:09.543403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:12.030 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~245s 2023-08-24T10:09:12.030 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~245s 2023-08-24T10:09:12.030 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~245s 2023-08-24T10:09:12.031 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~245s 2023-08-24T10:09:13.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:13.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.221+0000 7fd16d51e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:13.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.225+0000 7fd16d51e640 1 Processor -- start 2023-08-24T10:09:13.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.225+0000 7fd16d51e640 1 -- start start 2023-08-24T10:09:13.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.226+0000 7fd16d51e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd168131d40 0x7fd168134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:13.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.226+0000 7fd16d51e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd168099440 con 0x7fd16806d300 2023-08-24T10:09:13.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.226+0000 7fd16d51e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd16809a050 con 0x7fd168131d40 2023-08-24T10:09:13.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.226+0000 7fd15ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd168131d40 0x7fd168134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:13.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.226+0000 7fd15ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd168131d40 0x7fd168134160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35792/0 (socket says 172.21.15.60:35792) 2023-08-24T10:09:13.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.226+0000 7fd15ffff640 1 -- 172.21.15.60:0/3922750837 learned_addr learned my addr 172.21.15.60:0/3922750837 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:13.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.226+0000 7fd15ffff640 1 -- 172.21.15.60:0/3922750837 >> v1:172.21.15.60:6789/0 conn(0x7fd16806d300 legacy=0x7fd16812bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:13.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.226+0000 7fd15ffff640 1 -- 172.21.15.60:0/3922750837 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16812b9d0 con 0x7fd168131d40 2023-08-24T10:09:13.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.227+0000 7fd15ffff640 1 --2- 172.21.15.60:0/3922750837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd168131d40 0x7fd168134160 crc :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1197b363b6d0275 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:13.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.227+0000 7fd1667fc640 1 -- 172.21.15.60:0/3922750837 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd150009dc0 con 0x7fd168131d40 2023-08-24T10:09:13.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.227+0000 7fd1667fc640 1 -- 172.21.15.60:0/3922750837 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd15000f340 con 0x7fd168131d40 2023-08-24T10:09:13.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.227+0000 7fd1667fc640 1 -- 172.21.15.60:0/3922750837 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd15000f5e0 con 0x7fd168131d40 2023-08-24T10:09:13.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.227+0000 7fd16d51e640 1 -- 172.21.15.60:0/3922750837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd168131d40 msgr2=0x7fd168134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.227+0000 7fd16d51e640 1 --2- 172.21.15.60:0/3922750837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd168131d40 0x7fd168134160 crc :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.227+0000 7fd16d51e640 1 -- 172.21.15.60:0/3922750837 shutdown_connections 2023-08-24T10:09:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.227+0000 7fd16d51e640 1 --2- 172.21.15.60:0/3922750837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd168131d40 0x7fd168134160 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.228+0000 7fd16d51e640 1 -- 172.21.15.60:0/3922750837 >> 172.21.15.60:0/3922750837 conn(0x7fd16809e110 msgr2=0x7fd16809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.228+0000 7fd16d51e640 1 -- 172.21.15.60:0/3922750837 shutdown_connections 2023-08-24T10:09:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.228+0000 7fd16d51e640 1 -- 172.21.15.60:0/3922750837 wait complete. 2023-08-24T10:09:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.228+0000 7fd16d51e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:13.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.229+0000 7fd16d51e640 1 Processor -- start 2023-08-24T10:09:13.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.229+0000 7fd16d51e640 1 -- start start 2023-08-24T10:09:13.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.229+0000 7fd16d51e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd16806d300 0x7fd1680967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:13.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.229+0000 7fd16d51e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16806fe20 con 0x7fd16806d300 2023-08-24T10:09:13.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.229+0000 7fd166ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd16806d300 0x7fd1680967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:13.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.229+0000 7fd166ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd16806d300 0x7fd1680967e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35800/0 (socket says 172.21.15.60:35800) 2023-08-24T10:09:13.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.229+0000 7fd166ffd640 1 -- 172.21.15.60:0/3229271585 learned_addr learned my addr 172.21.15.60:0/3229271585 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:13.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.230+0000 7fd166ffd640 1 -- 172.21.15.60:0/3229271585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16812c930 con 0x7fd16806d300 2023-08-24T10:09:13.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.230+0000 7fd166ffd640 1 --2- 172.21.15.60:0/3229271585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd16806d300 0x7fd1680967e0 crc :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:13.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.230+0000 7fd15f7fe640 1 -- 172.21.15.60:0/3229271585 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd15800c860 con 0x7fd16806d300 2023-08-24T10:09:13.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.230+0000 7fd15f7fe640 1 -- 172.21.15.60:0/3229271585 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd15800c9c0 con 0x7fd16806d300 2023-08-24T10:09:13.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.230+0000 7fd15f7fe640 1 -- 172.21.15.60:0/3229271585 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd15800cc60 con 0x7fd16806d300 2023-08-24T10:09:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.230+0000 7fd16d51e640 1 -- 172.21.15.60:0/3229271585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16806b0e0 con 0x7fd16806d300 2023-08-24T10:09:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.230+0000 7fd16d51e640 1 -- 172.21.15.60:0/3229271585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd16806e980 con 0x7fd16806d300 2023-08-24T10:09:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.231+0000 7fd15f7fe640 1 -- 172.21.15.60:0/3229271585 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fd158011310 con 0x7fd16806d300 2023-08-24T10:09:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.231+0000 7fd16d51e640 1 -- 172.21.15.60:0/3229271585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd168061800 con 0x7fd16806d300 2023-08-24T10:09:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.231+0000 7fd15f7fe640 1 --2- 172.21.15.60:0/3229271585 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd148037130 0x7fd148039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.231+0000 7fd15f7fe640 1 -- 172.21.15.60:0/3229271585 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fd158046950 con 0x7fd16806d300 2023-08-24T10:09:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.232+0000 7fd15ffff640 1 --2- 172.21.15.60:0/3229271585 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd148037130 0x7fd148039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.232+0000 7fd15ffff640 1 --2- 172.21.15.60:0/3229271585 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd148037130 0x7fd148039640 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:13.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.236+0000 7fd15f7fe640 1 -- 172.21.15.60:0/3229271585 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fd158011e60 con 0x7fd16806d300 2023-08-24T10:09:13.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.407+0000 7fd16d51e640 1 -- 172.21.15.60:0/3229271585 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd16812a310 con 0x7fd148037130 2023-08-24T10:09:13.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.408+0000 7fd15f7fe640 1 -- 172.21.15.60:0/3229271585 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7fd150009d90 con 0x7fd148037130 2023-08-24T10:09:13.410 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:13.410 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:13.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.410+0000 7fd16d51e640 1 -- 172.21.15.60:0/3229271585 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd148037130 msgr2=0x7fd148039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:13.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.410+0000 7fd16d51e640 1 --2- 172.21.15.60:0/3229271585 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd148037130 0x7fd148039640 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:13.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.410+0000 7fd16d51e640 1 -- 172.21.15.60:0/3229271585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd16806d300 msgr2=0x7fd1680967e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:13.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.410+0000 7fd16d51e640 1 --2- 172.21.15.60:0/3229271585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd16806d300 0x7fd1680967e0 crc :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:13.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.410+0000 7fd16d51e640 1 -- 172.21.15.60:0/3229271585 shutdown_connections 2023-08-24T10:09:13.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.410+0000 7fd16d51e640 1 --2- 172.21.15.60:0/3229271585 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd148037130 0x7fd148039640 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:13.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.410+0000 7fd16d51e640 1 --2- 172.21.15.60:0/3229271585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd16806d300 0x7fd1680967e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:13.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.410+0000 7fd16d51e640 1 -- 172.21.15.60:0/3229271585 >> 172.21.15.60:0/3229271585 conn(0x7fd16809e110 msgr2=0x7fd16812ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:13.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.410+0000 7fd16d51e640 1 -- 172.21.15.60:0/3229271585 shutdown_connections 2023-08-24T10:09:13.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.410+0000 7fd16d51e640 1 -- 172.21.15.60:0/3229271585 wait complete. 2023-08-24T10:09:13.423 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2023-08-24T10:09:11.543924+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:13.424 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:13.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.531+0000 7f477f11e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:13.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.535+0000 7f477f11e640 1 Processor -- start 2023-08-24T10:09:13.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.535+0000 7f477f11e640 1 -- start start 2023-08-24T10:09:13.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.535+0000 7f477f11e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4778070110 0x7f47781356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:13.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.536+0000 7f477f11e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4778096f10 con 0x7f4778071e40 2023-08-24T10:09:13.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.536+0000 7f477f11e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47780977f0 con 0x7f4778070110 2023-08-24T10:09:13.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.536+0000 7f4777fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4778070110 0x7f47781356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:13.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.536+0000 7f4777fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4778070110 0x7f47781356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35808/0 (socket says 172.21.15.60:35808) 2023-08-24T10:09:13.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.536+0000 7f477ceba640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4778071e40 0x7f4778096800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34858/0 (socket says 172.21.15.60:34858) 2023-08-24T10:09:13.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.536+0000 7f4777fff640 1 -- 172.21.15.60:0/3022245201 learned_addr learned my addr 172.21.15.60:0/3022245201 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:13.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.536+0000 7f4777fff640 1 -- 172.21.15.60:0/3022245201 >> v1:172.21.15.60:6789/0 conn(0x7f4778071e40 legacy=0x7f4778096800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:13.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.536+0000 7f4777fff640 1 -- 172.21.15.60:0/3022245201 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477809a280 con 0x7f4778070110 2023-08-24T10:09:13.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.536+0000 7f4777fff640 1 --2- 172.21.15.60:0/3022245201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4778070110 0x7f47781356a0 crc :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc25943b3dba6bf2 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:13.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.536+0000 7f47777fe640 1 -- 172.21.15.60:0/3022245201 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4764009dd0 con 0x7f4778070110 2023-08-24T10:09:13.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.536+0000 7f47777fe640 1 -- 172.21.15.60:0/3022245201 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f476400e230 con 0x7f4778070110 2023-08-24T10:09:13.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.536+0000 7f47777fe640 1 -- 172.21.15.60:0/3022245201 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f476400e4d0 con 0x7f4778070110 2023-08-24T10:09:13.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.537+0000 7f477f11e640 1 -- 172.21.15.60:0/3022245201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4778070110 msgr2=0x7f47781356a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:13.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.537+0000 7f477f11e640 1 --2- 172.21.15.60:0/3022245201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4778070110 0x7f47781356a0 crc :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:13.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.537+0000 7f477f11e640 1 -- 172.21.15.60:0/3022245201 shutdown_connections 2023-08-24T10:09:13.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.537+0000 7f477f11e640 1 --2- 172.21.15.60:0/3022245201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4778070110 0x7f47781356a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:13.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.537+0000 7f477f11e640 1 -- 172.21.15.60:0/3022245201 >> 172.21.15.60:0/3022245201 conn(0x7f477809e110 msgr2=0x7f477809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:13.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.537+0000 7f477f11e640 1 -- 172.21.15.60:0/3022245201 shutdown_connections 2023-08-24T10:09:13.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.537+0000 7f477f11e640 1 -- 172.21.15.60:0/3022245201 wait complete. 2023-08-24T10:09:13.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.537+0000 7f477f11e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:13.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.538+0000 7f477f11e640 1 Processor -- start 2023-08-24T10:09:13.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.538+0000 7f477f11e640 1 -- start start 2023-08-24T10:09:13.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.539+0000 7f477f11e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4778070110 0x7f4778093460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:13.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.539+0000 7f477f11e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477808ff10 con 0x7f4778070110 2023-08-24T10:09:13.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.539+0000 7f477ceba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4778070110 0x7f4778093460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:13.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.539+0000 7f477ceba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4778070110 0x7f4778093460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35814/0 (socket says 172.21.15.60:35814) 2023-08-24T10:09:13.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.539+0000 7f477ceba640 1 -- 172.21.15.60:0/973715502 learned_addr learned my addr 172.21.15.60:0/973715502 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:13.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.539+0000 7f477ceba640 1 -- 172.21.15.60:0/973715502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4778090680 con 0x7f4778070110 2023-08-24T10:09:13.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.539+0000 7f477ceba640 1 --2- 172.21.15.60:0/973715502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4778070110 0x7f4778093460 crc :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:13.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.540+0000 7f47757fa640 1 -- 172.21.15.60:0/973715502 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f476c0071e0 con 0x7f4778070110 2023-08-24T10:09:13.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.540+0000 7f47757fa640 1 -- 172.21.15.60:0/973715502 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f476c007340 con 0x7f4778070110 2023-08-24T10:09:13.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.540+0000 7f477f11e640 1 -- 172.21.15.60:0/973715502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4778092e80 con 0x7f4778070110 2023-08-24T10:09:13.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.540+0000 7f47757fa640 1 -- 172.21.15.60:0/973715502 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f476c0075e0 con 0x7f4778070110 2023-08-24T10:09:13.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.540+0000 7f477f11e640 1 -- 172.21.15.60:0/973715502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4778092b70 con 0x7f4778070110 2023-08-24T10:09:13.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.541+0000 7f47757fa640 1 -- 172.21.15.60:0/973715502 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f476c0057e0 con 0x7f4778070110 2023-08-24T10:09:13.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.541+0000 7f477f11e640 1 -- 172.21.15.60:0/973715502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4778061800 con 0x7f4778070110 2023-08-24T10:09:13.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.541+0000 7f47757fa640 1 --2- 172.21.15.60:0/973715502 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f475c037130 0x7f475c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:13.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.541+0000 7f47757fa640 1 -- 172.21.15.60:0/973715502 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f476c0463f0 con 0x7f4778070110 2023-08-24T10:09:13.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.541+0000 7f4777fff640 1 --2- 172.21.15.60:0/973715502 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f475c037130 0x7f475c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:13.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.542+0000 7f4777fff640 1 --2- 172.21.15.60:0/973715502 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f475c037130 0x7f475c039640 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:13.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.546+0000 7f47757fa640 1 -- 172.21.15.60:0/973715502 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f476c00d330 con 0x7f4778070110 2023-08-24T10:09:13.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.717+0000 7f477f11e640 1 -- 172.21.15.60:0/973715502 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f47780932f0 con 0x7f475c037130 2023-08-24T10:09:13.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.719+0000 7f47757fa640 1 -- 172.21.15.60:0/973715502 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f47780932f0 con 0x7f475c037130 2023-08-24T10:09:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:13.720 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:13.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.720+0000 7f477f11e640 1 -- 172.21.15.60:0/973715502 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f475c037130 msgr2=0x7f475c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.720+0000 7f477f11e640 1 --2- 172.21.15.60:0/973715502 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f475c037130 0x7f475c039640 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.720+0000 7f477f11e640 1 -- 172.21.15.60:0/973715502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4778070110 msgr2=0x7f4778093460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.720+0000 7f477f11e640 1 --2- 172.21.15.60:0/973715502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4778070110 0x7f4778093460 crc :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.720+0000 7f477f11e640 1 -- 172.21.15.60:0/973715502 shutdown_connections 2023-08-24T10:09:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.721+0000 7f477f11e640 1 --2- 172.21.15.60:0/973715502 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f475c037130 0x7f475c039640 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.721+0000 7f477f11e640 1 --2- 172.21.15.60:0/973715502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4778070110 0x7f4778093460 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.721+0000 7f477f11e640 1 -- 172.21.15.60:0/973715502 >> 172.21.15.60:0/973715502 conn(0x7f477809e110 msgr2=0x7f4778098490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.721+0000 7f477f11e640 1 -- 172.21.15.60:0/973715502 shutdown_connections 2023-08-24T10:09:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.721+0000 7f477f11e640 1 -- 172.21.15.60:0/973715502 wait complete. 2023-08-24T10:09:13.732 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2023-08-24T10:09:13.544287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:13.733 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:09:13.733 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:13.734 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:09:13.734 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:13.734 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:09:13.734 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:13.735 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:09:13.735 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:13.735 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:09:13.735 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:13.736 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:09:13.736 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:13.736 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:09:13.736 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:13.737 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:09:13.737 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:13.737 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:09:13.737 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:13.738 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:09:13.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.845+0000 7f8c2e630640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:13.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.848+0000 7f8c2e630640 1 Processor -- start 2023-08-24T10:09:13.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.848+0000 7f8c2e630640 1 -- start start 2023-08-24T10:09:13.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.849+0000 7f8c2e630640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8c28070c30 0x7f8c28071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:13.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.849+0000 7f8c2e630640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c2806dfc0 con 0x7f8c280990b0 2023-08-24T10:09:13.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.849+0000 7f8c2e630640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c2806d9b0 con 0x7f8c28070c30 2023-08-24T10:09:13.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.849+0000 7f8c27fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8c28070c30 0x7f8c28071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:13.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.849+0000 7f8c27fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8c28070c30 0x7f8c28071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35818/0 (socket says 172.21.15.60:35818) 2023-08-24T10:09:13.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.849+0000 7f8c27fff640 1 -- 172.21.15.60:0/3048788188 learned_addr learned my addr 172.21.15.60:0/3048788188 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:13.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.849+0000 7f8c277fe640 1 --1- 172.21.15.60:0/3048788188 >> v1:172.21.15.60:6789/0 conn(0x7f8c280990b0 0x7f8c280994c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34860/0 (socket says 172.21.15.60:34860) 2023-08-24T10:09:13.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.849+0000 7f8c27fff640 1 -- 172.21.15.60:0/3048788188 >> v1:172.21.15.60:6789/0 conn(0x7f8c280990b0 legacy=0x7f8c280994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:13.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.849+0000 7f8c27fff640 1 -- 172.21.15.60:0/3048788188 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2806e6e0 con 0x7f8c28070c30 2023-08-24T10:09:13.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.849+0000 7f8c27fff640 1 --2- 172.21.15.60:0/3048788188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28070c30 0x7f8c28071040 crc :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98776ecf303c9e6b server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:13.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.849+0000 7f8c26ffd640 1 -- 172.21.15.60:0/3048788188 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8c18009dd0 con 0x7f8c28070c30 2023-08-24T10:09:13.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.849+0000 7f8c26ffd640 1 -- 172.21.15.60:0/3048788188 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f8c1800e230 con 0x7f8c28070c30 2023-08-24T10:09:13.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.849+0000 7f8c26ffd640 1 -- 172.21.15.60:0/3048788188 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8c1800e4d0 con 0x7f8c28070c30 2023-08-24T10:09:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.850+0000 7f8c2e630640 1 -- 172.21.15.60:0/3048788188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28070c30 msgr2=0x7f8c28071040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.850+0000 7f8c2e630640 1 --2- 172.21.15.60:0/3048788188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28070c30 0x7f8c28071040 crc :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.850+0000 7f8c2e630640 1 -- 172.21.15.60:0/3048788188 shutdown_connections 2023-08-24T10:09:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.850+0000 7f8c2e630640 1 --2- 172.21.15.60:0/3048788188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28070c30 0x7f8c28071040 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.850+0000 7f8c2e630640 1 -- 172.21.15.60:0/3048788188 >> 172.21.15.60:0/3048788188 conn(0x7f8c2809e110 msgr2=0x7f8c2809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.850+0000 7f8c2e630640 1 -- 172.21.15.60:0/3048788188 shutdown_connections 2023-08-24T10:09:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.850+0000 7f8c2e630640 1 -- 172.21.15.60:0/3048788188 wait complete. 2023-08-24T10:09:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.850+0000 7f8c2e630640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.851+0000 7f8c2e630640 1 Processor -- start 2023-08-24T10:09:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.851+0000 7f8c2e630640 1 -- start start 2023-08-24T10:09:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.852+0000 7f8c2e630640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28070c30 0x7f8c281c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.852+0000 7f8c2e630640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2812b3e0 con 0x7f8c28070c30 2023-08-24T10:09:13.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.852+0000 7f8c27fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28070c30 0x7f8c281c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:13.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.852+0000 7f8c27fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28070c30 0x7f8c281c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35826/0 (socket says 172.21.15.60:35826) 2023-08-24T10:09:13.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.852+0000 7f8c27fff640 1 -- 172.21.15.60:0/1387795305 learned_addr learned my addr 172.21.15.60:0/1387795305 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:13.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.852+0000 7f8c27fff640 1 -- 172.21.15.60:0/1387795305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c281c0930 con 0x7f8c28070c30 2023-08-24T10:09:13.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.853+0000 7f8c27fff640 1 --2- 172.21.15.60:0/1387795305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28070c30 0x7f8c281c0fa0 crc :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:13.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.853+0000 7f8c24ff9640 1 -- 172.21.15.60:0/1387795305 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8c18014410 con 0x7f8c28070c30 2023-08-24T10:09:13.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.853+0000 7f8c24ff9640 1 -- 172.21.15.60:0/1387795305 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f8c18014570 con 0x7f8c28070c30 2023-08-24T10:09:13.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.853+0000 7f8c2e630640 1 -- 172.21.15.60:0/1387795305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c281c0c00 con 0x7f8c28070c30 2023-08-24T10:09:13.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.853+0000 7f8c24ff9640 1 -- 172.21.15.60:0/1387795305 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8c18014810 con 0x7f8c28070c30 2023-08-24T10:09:13.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.853+0000 7f8c2e630640 1 -- 172.21.15.60:0/1387795305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2812c980 con 0x7f8c28070c30 2023-08-24T10:09:13.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.854+0000 7f8c24ff9640 1 -- 172.21.15.60:0/1387795305 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f8c180149c0 con 0x7f8c28070c30 2023-08-24T10:09:13.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.854+0000 7f8c24ff9640 1 --2- 172.21.15.60:0/1387795305 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8c14037130 0x7f8c14039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:13.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.854+0000 7f8c24ff9640 1 -- 172.21.15.60:0/1387795305 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f8c18019040 con 0x7f8c28070c30 2023-08-24T10:09:13.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.854+0000 7f8c2e630640 1 -- 172.21.15.60:0/1387795305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2812dd10 con 0x7f8c28070c30 2023-08-24T10:09:13.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.854+0000 7f8c277fe640 1 --2- 172.21.15.60:0/1387795305 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8c14037130 0x7f8c14039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:13.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.855+0000 7f8c277fe640 1 --2- 172.21.15.60:0/1387795305 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8c14037130 0x7f8c14039640 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:13.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:13.859+0000 7f8c24ff9640 1 -- 172.21.15.60:0/1387795305 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f8c1800fb20 con 0x7f8c28070c30 2023-08-24T10:09:14.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.102+0000 7f8c2e630640 1 -- 172.21.15.60:0/1387795305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8c2812c1b0 con 0x7f8c28070c30 2023-08-24T10:09:14.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.103+0000 7f8c24ff9640 1 -- 172.21.15.60:0/1387795305 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f8c18005960 con 0x7f8c28070c30 2023-08-24T10:09:14.159 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:14.160 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":264,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:09:14.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.104+0000 7f8c2e630640 1 -- 172.21.15.60:0/1387795305 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8c14037130 msgr2=0x7f8c14039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:14.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.104+0000 7f8c2e630640 1 --2- 172.21.15.60:0/1387795305 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8c14037130 0x7f8c14039640 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:14.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.104+0000 7f8c2e630640 1 -- 172.21.15.60:0/1387795305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28070c30 msgr2=0x7f8c281c0fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.104+0000 7f8c2e630640 1 --2- 172.21.15.60:0/1387795305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28070c30 0x7f8c281c0fa0 crc :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.104+0000 7f8c2e630640 1 -- 172.21.15.60:0/1387795305 shutdown_connections 2023-08-24T10:09:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.104+0000 7f8c2e630640 1 --2- 172.21.15.60:0/1387795305 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8c14037130 0x7f8c14039640 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.104+0000 7f8c2e630640 1 --2- 172.21.15.60:0/1387795305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c28070c30 0x7f8c281c0fa0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.104+0000 7f8c2e630640 1 -- 172.21.15.60:0/1387795305 >> 172.21.15.60:0/1387795305 conn(0x7f8c2809e110 msgr2=0x7f8c281b75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.105+0000 7f8c2e630640 1 -- 172.21.15.60:0/1387795305 shutdown_connections 2023-08-24T10:09:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.105+0000 7f8c2e630640 1 -- 172.21.15.60:0/1387795305 wait complete. 2023-08-24T10:09:14.163 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:09:14.163 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:14.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.270+0000 7f271679e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:14.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.274+0000 7f271679e640 1 Processor -- start 2023-08-24T10:09:14.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.274+0000 7f271679e640 1 -- start start 2023-08-24T10:09:14.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.275+0000 7f271679e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f27100a1c70 0x7f271012d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:14.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.275+0000 7f271679e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f271006e170 con 0x7f271012d6a0 2023-08-24T10:09:14.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.275+0000 7f271679e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27100a2080 con 0x7f27100a1c70 2023-08-24T10:09:14.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.275+0000 7f270ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f27100a1c70 0x7f271012d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:14.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.275+0000 7f270ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f27100a1c70 0x7f271012d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35840/0 (socket says 172.21.15.60:35840) 2023-08-24T10:09:14.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.275+0000 7f270ffff640 1 -- 172.21.15.60:0/3637873895 learned_addr learned my addr 172.21.15.60:0/3637873895 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:14.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.275+0000 7f270f7fe640 1 --1- 172.21.15.60:0/3637873895 >> v1:172.21.15.60:6789/0 conn(0x7f271012d6a0 0x7f2710069bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34872/0 (socket says 172.21.15.60:34872) 2023-08-24T10:09:14.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.275+0000 7f270ffff640 1 -- 172.21.15.60:0/3637873895 >> v1:172.21.15.60:6789/0 conn(0x7f271012d6a0 legacy=0x7f2710069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:14.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.275+0000 7f270ffff640 1 -- 172.21.15.60:0/3637873895 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271012cec0 con 0x7f27100a1c70 2023-08-24T10:09:14.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.275+0000 7f270ffff640 1 --2- 172.21.15.60:0/3637873895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27100a1c70 0x7f271012d160 crc :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e3833b1fec6e57 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:14.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.275+0000 7f270effd640 1 -- 172.21.15.60:0/3637873895 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f26f8009dd0 con 0x7f27100a1c70 2023-08-24T10:09:14.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.276+0000 7f270effd640 1 -- 172.21.15.60:0/3637873895 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f26f800e230 con 0x7f27100a1c70 2023-08-24T10:09:14.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.276+0000 7f270effd640 1 -- 172.21.15.60:0/3637873895 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f26f800e4d0 con 0x7f27100a1c70 2023-08-24T10:09:14.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.276+0000 7f271679e640 1 -- 172.21.15.60:0/3637873895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27100a1c70 msgr2=0x7f271012d160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:14.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.276+0000 7f271679e640 1 --2- 172.21.15.60:0/3637873895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27100a1c70 0x7f271012d160 crc :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:14.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.276+0000 7f271679e640 1 -- 172.21.15.60:0/3637873895 shutdown_connections 2023-08-24T10:09:14.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.276+0000 7f271679e640 1 --2- 172.21.15.60:0/3637873895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27100a1c70 0x7f271012d160 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:14.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.276+0000 7f271679e640 1 -- 172.21.15.60:0/3637873895 >> 172.21.15.60:0/3637873895 conn(0x7f271009f290 msgr2=0x7f271009f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:14.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.276+0000 7f271679e640 1 -- 172.21.15.60:0/3637873895 shutdown_connections 2023-08-24T10:09:14.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.276+0000 7f271679e640 1 -- 172.21.15.60:0/3637873895 wait complete. 2023-08-24T10:09:14.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.276+0000 7f271679e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:14.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.277+0000 7f271679e640 1 Processor -- start 2023-08-24T10:09:14.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.277+0000 7f271679e640 1 -- start start 2023-08-24T10:09:14.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.278+0000 7f271679e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27100a1c70 0x7f2710130640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:14.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.278+0000 7f271679e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271012ffc0 con 0x7f27100a1c70 2023-08-24T10:09:14.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.278+0000 7f270ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27100a1c70 0x7f2710130640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:14.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.278+0000 7f270ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27100a1c70 0x7f2710130640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35852/0 (socket says 172.21.15.60:35852) 2023-08-24T10:09:14.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.278+0000 7f270ffff640 1 -- 172.21.15.60:0/3203042511 learned_addr learned my addr 172.21.15.60:0/3203042511 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:14.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.278+0000 7f270ffff640 1 -- 172.21.15.60:0/3203042511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271012a7e0 con 0x7f27100a1c70 2023-08-24T10:09:14.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.279+0000 7f270ffff640 1 --2- 172.21.15.60:0/3203042511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27100a1c70 0x7f2710130640 crc :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:14.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.279+0000 7f270cff9640 1 -- 172.21.15.60:0/3203042511 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f26f8014540 con 0x7f27100a1c70 2023-08-24T10:09:14.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.279+0000 7f270cff9640 1 -- 172.21.15.60:0/3203042511 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f26f80146a0 con 0x7f27100a1c70 2023-08-24T10:09:14.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.279+0000 7f271679e640 1 -- 172.21.15.60:0/3203042511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271012fc20 con 0x7f27100a1c70 2023-08-24T10:09:14.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.279+0000 7f270cff9640 1 -- 172.21.15.60:0/3203042511 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f26f8014940 con 0x7f27100a1c70 2023-08-24T10:09:14.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.279+0000 7f271679e640 1 -- 172.21.15.60:0/3203042511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2710119ca0 con 0x7f27100a1c70 2023-08-24T10:09:14.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.280+0000 7f270cff9640 1 -- 172.21.15.60:0/3203042511 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f26f8014af0 con 0x7f27100a1c70 2023-08-24T10:09:14.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.280+0000 7f271679e640 1 -- 172.21.15.60:0/3203042511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2710117c30 con 0x7f27100a1c70 2023-08-24T10:09:14.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.280+0000 7f270cff9640 1 --2- 172.21.15.60:0/3203042511 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f26f0037130 0x7f26f0039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:14.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.280+0000 7f270cff9640 1 -- 172.21.15.60:0/3203042511 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f26f8019040 con 0x7f27100a1c70 2023-08-24T10:09:14.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.280+0000 7f270f7fe640 1 --2- 172.21.15.60:0/3203042511 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f26f0037130 0x7f26f0039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:14.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.281+0000 7f270f7fe640 1 --2- 172.21.15.60:0/3203042511 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f26f0037130 0x7f26f0039640 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:14.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.284+0000 7f270cff9640 1 -- 172.21.15.60:0/3203042511 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f26f8018190 con 0x7f27100a1c70 2023-08-24T10:09:14.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.463+0000 7f271679e640 1 -- 172.21.15.60:0/3203042511 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f271011a2c0 con 0x7f26f0037130 2023-08-24T10:09:14.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.464+0000 7f270cff9640 1 -- 172.21.15.60:0/3203042511 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f271011a2c0 con 0x7f26f0037130 2023-08-24T10:09:14.465 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:14.466 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:14.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.466+0000 7f271679e640 1 -- 172.21.15.60:0/3203042511 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f26f0037130 msgr2=0x7f26f0039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:14.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.466+0000 7f271679e640 1 --2- 172.21.15.60:0/3203042511 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f26f0037130 0x7f26f0039640 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:14.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.466+0000 7f271679e640 1 -- 172.21.15.60:0/3203042511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27100a1c70 msgr2=0x7f2710130640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:14.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.466+0000 7f271679e640 1 --2- 172.21.15.60:0/3203042511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27100a1c70 0x7f2710130640 crc :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:14.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.466+0000 7f271679e640 1 -- 172.21.15.60:0/3203042511 shutdown_connections 2023-08-24T10:09:14.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.466+0000 7f271679e640 1 --2- 172.21.15.60:0/3203042511 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f26f0037130 0x7f26f0039640 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:14.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.466+0000 7f271679e640 1 --2- 172.21.15.60:0/3203042511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27100a1c70 0x7f2710130640 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:14.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.466+0000 7f271679e640 1 -- 172.21.15.60:0/3203042511 >> 172.21.15.60:0/3203042511 conn(0x7f271009f290 msgr2=0x7f27101b8440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:14.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.466+0000 7f271679e640 1 -- 172.21.15.60:0/3203042511 shutdown_connections 2023-08-24T10:09:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:14.466+0000 7f271679e640 1 -- 172.21.15.60:0/3203042511 wait complete. 2023-08-24T10:09:14.481 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2023-08-24T10:09:13.544287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:17.233 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~250s 2023-08-24T10:09:17.234 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~250s 2023-08-24T10:09:17.234 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~250s 2023-08-24T10:09:17.234 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~250s 2023-08-24T10:09:17.483 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:17.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.603+0000 7f584c39e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:17.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.607+0000 7f584c39e640 1 Processor -- start 2023-08-24T10:09:17.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.607+0000 7f584c39e640 1 -- start start 2023-08-24T10:09:17.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.608+0000 7f584c39e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5844132b50 0x7f5844132f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:17.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.608+0000 7f584c39e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5844132610 con 0x7f584412f110 2023-08-24T10:09:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.608+0000 7f584c39e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f584409c260 con 0x7f5844132b50 2023-08-24T10:09:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.608+0000 7f584a13a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5844132b50 0x7f5844132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.608+0000 7f5849939640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f584412f110 0x7f584412f520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38752/0 (socket says 172.21.15.60:38752) 2023-08-24T10:09:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.608+0000 7f584a13a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5844132b50 0x7f5844132f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34042/0 (socket says 172.21.15.60:34042) 2023-08-24T10:09:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.608+0000 7f5849939640 1 -- 172.21.15.60:0/1363387297 learned_addr learned my addr 172.21.15.60:0/1363387297 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:09:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.609+0000 7f584a13a640 1 -- 172.21.15.60:0/1363387297 >> v1:172.21.15.60:6789/0 conn(0x7f584412f110 legacy=0x7f584412f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:17.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.609+0000 7f584a13a640 1 -- 172.21.15.60:0/1363387297 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5844132840 con 0x7f5844132b50 2023-08-24T10:09:17.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.609+0000 7f584a13a640 1 --2- 172.21.15.60:0/1363387297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5844132b50 0x7f5844132f60 crc :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c44f0ccb32e719 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:17.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.609+0000 7f5849138640 1 -- 172.21.15.60:0/1363387297 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5840009ce0 con 0x7f5844132b50 2023-08-24T10:09:17.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.609+0000 7f5849138640 1 -- 172.21.15.60:0/1363387297 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5840009e40 con 0x7f5844132b50 2023-08-24T10:09:17.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.609+0000 7f5849138640 1 -- 172.21.15.60:0/1363387297 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f584000e360 con 0x7f5844132b50 2023-08-24T10:09:17.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.610+0000 7f584c39e640 1 -- 172.21.15.60:0/1363387297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5844132b50 msgr2=0x7f5844132f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:17.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.610+0000 7f584c39e640 1 --2- 172.21.15.60:0/1363387297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5844132b50 0x7f5844132f60 crc :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:17.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.610+0000 7f584c39e640 1 -- 172.21.15.60:0/1363387297 shutdown_connections 2023-08-24T10:09:17.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.610+0000 7f584c39e640 1 --2- 172.21.15.60:0/1363387297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5844132b50 0x7f5844132f60 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:17.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.610+0000 7f584c39e640 1 -- 172.21.15.60:0/1363387297 >> 172.21.15.60:0/1363387297 conn(0x7f58440a26a0 msgr2=0x7f58440a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:17.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.610+0000 7f584c39e640 1 -- 172.21.15.60:0/1363387297 shutdown_connections 2023-08-24T10:09:17.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.610+0000 7f584c39e640 1 -- 172.21.15.60:0/1363387297 wait complete. 2023-08-24T10:09:17.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.610+0000 7f584c39e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:17.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.611+0000 7f584c39e640 1 Processor -- start 2023-08-24T10:09:17.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.611+0000 7f584c39e640 1 -- start start 2023-08-24T10:09:17.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.612+0000 7f584c39e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584412f110 0x7f584409fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:17.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.612+0000 7f584c39e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584412cc80 con 0x7f584412f110 2023-08-24T10:09:17.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.612+0000 7f584a13a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584412f110 0x7f584409fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:17.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.612+0000 7f584a13a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584412f110 0x7f584409fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34054/0 (socket says 172.21.15.60:34054) 2023-08-24T10:09:17.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.612+0000 7f584a13a640 1 -- 172.21.15.60:0/1105185097 learned_addr learned my addr 172.21.15.60:0/1105185097 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:17.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.612+0000 7f584a13a640 1 -- 172.21.15.60:0/1105185097 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584409f400 con 0x7f584412f110 2023-08-24T10:09:17.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.612+0000 7f584a13a640 1 --2- 172.21.15.60:0/1105185097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584412f110 0x7f584409fcd0 crc :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:17.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.612+0000 7f583affd640 1 -- 172.21.15.60:0/1105185097 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5840014620 con 0x7f584412f110 2023-08-24T10:09:17.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.613+0000 7f583affd640 1 -- 172.21.15.60:0/1105185097 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5840014780 con 0x7f584412f110 2023-08-24T10:09:17.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.613+0000 7f584c39e640 1 -- 172.21.15.60:0/1105185097 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58440a18e0 con 0x7f584412f110 2023-08-24T10:09:17.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.613+0000 7f583affd640 1 -- 172.21.15.60:0/1105185097 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5840014a20 con 0x7f584412f110 2023-08-24T10:09:17.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.613+0000 7f584c39e640 1 -- 172.21.15.60:0/1105185097 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584406a1d0 con 0x7f584412f110 2023-08-24T10:09:17.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.613+0000 7f583affd640 1 -- 172.21.15.60:0/1105185097 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f5840014b80 con 0x7f584412f110 2023-08-24T10:09:17.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.614+0000 7f584c39e640 1 -- 172.21.15.60:0/1105185097 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584411c2b0 con 0x7f584412f110 2023-08-24T10:09:17.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.614+0000 7f583affd640 1 --2- 172.21.15.60:0/1105185097 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5828037130 0x7f5828039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:17.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.614+0000 7f583affd640 1 -- 172.21.15.60:0/1105185097 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f5840019040 con 0x7f584412f110 2023-08-24T10:09:17.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.614+0000 7f5849939640 1 --2- 172.21.15.60:0/1105185097 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5828037130 0x7f5828039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:17.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.614+0000 7f5849939640 1 --2- 172.21.15.60:0/1105185097 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5828037130 0x7f5828039640 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:17.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.619+0000 7f583affd640 1 -- 172.21.15.60:0/1105185097 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f5840018d90 con 0x7f584412f110 2023-08-24T10:09:17.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.793+0000 7f584c39e640 1 -- 172.21.15.60:0/1105185097 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58441334a0 con 0x7f5828037130 2023-08-24T10:09:17.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.794+0000 7f583affd640 1 -- 172.21.15.60:0/1105185097 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f5834001250 con 0x7f5828037130 2023-08-24T10:09:17.795 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:17.796 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.796+0000 7f584c39e640 1 -- 172.21.15.60:0/1105185097 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5828037130 msgr2=0x7f5828039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.796+0000 7f584c39e640 1 --2- 172.21.15.60:0/1105185097 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5828037130 0x7f5828039640 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.796+0000 7f584c39e640 1 -- 172.21.15.60:0/1105185097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584412f110 msgr2=0x7f584409fcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.796+0000 7f584c39e640 1 --2- 172.21.15.60:0/1105185097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584412f110 0x7f584409fcd0 crc :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.796+0000 7f584c39e640 1 -- 172.21.15.60:0/1105185097 shutdown_connections 2023-08-24T10:09:17.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.796+0000 7f584c39e640 1 --2- 172.21.15.60:0/1105185097 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5828037130 0x7f5828039640 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:17.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.796+0000 7f584c39e640 1 --2- 172.21.15.60:0/1105185097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f584412f110 0x7f584409fcd0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:17.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.796+0000 7f584c39e640 1 -- 172.21.15.60:0/1105185097 >> 172.21.15.60:0/1105185097 conn(0x7f58440a26a0 msgr2=0x7f58441ba920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:17.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.796+0000 7f584c39e640 1 -- 172.21.15.60:0/1105185097 shutdown_connections 2023-08-24T10:09:17.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.796+0000 7f584c39e640 1 -- 172.21.15.60:0/1105185097 wait complete. 2023-08-24T10:09:17.807 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2023-08-24T10:09:17.545117+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:17.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:17.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.914+0000 7fa0df11e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:17.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.918+0000 7fa0df11e640 1 Processor -- start 2023-08-24T10:09:17.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.918+0000 7fa0df11e640 1 -- start start 2023-08-24T10:09:17.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.919+0000 7fa0df11e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa0d8131d40 0x7fa0d8134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:17.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.919+0000 7fa0df11e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d8099440 con 0x7fa0d806d300 2023-08-24T10:09:17.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.919+0000 7fa0df11e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0d809a050 con 0x7fa0d8131d40 2023-08-24T10:09:17.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.919+0000 7fa0d7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa0d8131d40 0x7fa0d8134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:17.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.919+0000 7fa0d7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa0d8131d40 0x7fa0d8134160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34062/0 (socket says 172.21.15.60:34062) 2023-08-24T10:09:17.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.919+0000 7fa0dceba640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa0d806d300 0x7fa0d812bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38766/0 (socket says 172.21.15.60:38766) 2023-08-24T10:09:17.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.919+0000 7fa0d7fff640 1 -- 172.21.15.60:0/1649054471 learned_addr learned my addr 172.21.15.60:0/1649054471 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:17.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.920+0000 7fa0d7fff640 1 -- 172.21.15.60:0/1649054471 >> v1:172.21.15.60:6789/0 conn(0x7fa0d806d300 legacy=0x7fa0d812bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:17.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.920+0000 7fa0d7fff640 1 -- 172.21.15.60:0/1649054471 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d812b9d0 con 0x7fa0d8131d40 2023-08-24T10:09:17.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.920+0000 7fa0d7fff640 1 --2- 172.21.15.60:0/1649054471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d8131d40 0x7fa0d8134160 crc :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41caf0a9ddd19794 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:17.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.920+0000 7fa0d77fe640 1 -- 172.21.15.60:0/1649054471 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa0cc009dc0 con 0x7fa0d8131d40 2023-08-24T10:09:17.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.920+0000 7fa0d77fe640 1 -- 172.21.15.60:0/1649054471 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa0cc00e230 con 0x7fa0d8131d40 2023-08-24T10:09:17.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.920+0000 7fa0d77fe640 1 -- 172.21.15.60:0/1649054471 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa0cc00e4d0 con 0x7fa0d8131d40 2023-08-24T10:09:17.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.921+0000 7fa0df11e640 1 -- 172.21.15.60:0/1649054471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d8131d40 msgr2=0x7fa0d8134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:17.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.921+0000 7fa0df11e640 1 --2- 172.21.15.60:0/1649054471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d8131d40 0x7fa0d8134160 crc :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:17.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.921+0000 7fa0df11e640 1 -- 172.21.15.60:0/1649054471 shutdown_connections 2023-08-24T10:09:17.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.921+0000 7fa0df11e640 1 --2- 172.21.15.60:0/1649054471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d8131d40 0x7fa0d8134160 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:17.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.921+0000 7fa0df11e640 1 -- 172.21.15.60:0/1649054471 >> 172.21.15.60:0/1649054471 conn(0x7fa0d809e110 msgr2=0x7fa0d809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:17.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.921+0000 7fa0df11e640 1 -- 172.21.15.60:0/1649054471 shutdown_connections 2023-08-24T10:09:17.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.921+0000 7fa0df11e640 1 -- 172.21.15.60:0/1649054471 wait complete. 2023-08-24T10:09:17.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.921+0000 7fa0df11e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:17.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.922+0000 7fa0df11e640 1 Processor -- start 2023-08-24T10:09:17.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.922+0000 7fa0df11e640 1 -- start start 2023-08-24T10:09:17.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.923+0000 7fa0df11e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d806d300 0x7fa0d81274e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:17.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.923+0000 7fa0df11e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d8128f40 con 0x7fa0d806d300 2023-08-24T10:09:17.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.923+0000 7fa0dceba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d806d300 0x7fa0d81274e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:17.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.923+0000 7fa0dceba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d806d300 0x7fa0d81274e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34076/0 (socket says 172.21.15.60:34076) 2023-08-24T10:09:17.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.923+0000 7fa0dceba640 1 -- 172.21.15.60:0/116492172 learned_addr learned my addr 172.21.15.60:0/116492172 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:17.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.923+0000 7fa0dceba640 1 -- 172.21.15.60:0/116492172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d81286d0 con 0x7fa0d806d300 2023-08-24T10:09:17.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.923+0000 7fa0dceba640 1 --2- 172.21.15.60:0/116492172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d806d300 0x7fa0d81274e0 crc :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:17.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.923+0000 7fa0d57fa640 1 -- 172.21.15.60:0/116492172 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa0c80071e0 con 0x7fa0d806d300 2023-08-24T10:09:17.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.923+0000 7fa0d57fa640 1 -- 172.21.15.60:0/116492172 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa0c8007340 con 0x7fa0d806d300 2023-08-24T10:09:17.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.923+0000 7fa0df11e640 1 -- 172.21.15.60:0/116492172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d806b0e0 con 0x7fa0d806d300 2023-08-24T10:09:17.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.923+0000 7fa0d57fa640 1 -- 172.21.15.60:0/116492172 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa0c80075e0 con 0x7fa0d806d300 2023-08-24T10:09:17.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.923+0000 7fa0df11e640 1 -- 172.21.15.60:0/116492172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d8071aa0 con 0x7fa0d806d300 2023-08-24T10:09:17.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.924+0000 7fa0d57fa640 1 -- 172.21.15.60:0/116492172 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fa0c80057e0 con 0x7fa0d806d300 2023-08-24T10:09:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.924+0000 7fa0df11e640 1 -- 172.21.15.60:0/116492172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d806d9f0 con 0x7fa0d806d300 2023-08-24T10:09:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.924+0000 7fa0d57fa640 1 --2- 172.21.15.60:0/116492172 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa0c4037130 0x7fa0c4039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.924+0000 7fa0d57fa640 1 -- 172.21.15.60:0/116492172 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fa0c8046300 con 0x7fa0d806d300 2023-08-24T10:09:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.925+0000 7fa0d7fff640 1 --2- 172.21.15.60:0/116492172 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa0c4037130 0x7fa0c4039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.925+0000 7fa0d7fff640 1 --2- 172.21.15.60:0/116492172 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa0c4037130 0x7fa0c4039640 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:17.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:17.929+0000 7fa0d57fa640 1 -- 172.21.15.60:0/116492172 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fa0c800d330 con 0x7fa0d806d300 2023-08-24T10:09:18.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.099+0000 7fa0df11e640 1 -- 172.21.15.60:0/116492172 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0d8098880 con 0x7fa0c4037130 2023-08-24T10:09:18.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.101+0000 7fa0d57fa640 1 -- 172.21.15.60:0/116492172 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7fa0cc009d90 con 0x7fa0c4037130 2023-08-24T10:09:18.102 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:18.102 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:18.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.102+0000 7fa0df11e640 1 -- 172.21.15.60:0/116492172 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa0c4037130 msgr2=0x7fa0c4039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.102+0000 7fa0df11e640 1 --2- 172.21.15.60:0/116492172 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa0c4037130 0x7fa0c4039640 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.102+0000 7fa0df11e640 1 -- 172.21.15.60:0/116492172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d806d300 msgr2=0x7fa0d81274e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.102+0000 7fa0df11e640 1 --2- 172.21.15.60:0/116492172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d806d300 0x7fa0d81274e0 crc :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.102+0000 7fa0df11e640 1 -- 172.21.15.60:0/116492172 shutdown_connections 2023-08-24T10:09:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.102+0000 7fa0df11e640 1 --2- 172.21.15.60:0/116492172 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa0c4037130 0x7fa0c4039640 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.102+0000 7fa0df11e640 1 --2- 172.21.15.60:0/116492172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0d806d300 0x7fa0d81274e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.102+0000 7fa0df11e640 1 -- 172.21.15.60:0/116492172 >> 172.21.15.60:0/116492172 conn(0x7fa0d809e110 msgr2=0x7fa0d8129720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:18.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.102+0000 7fa0df11e640 1 -- 172.21.15.60:0/116492172 shutdown_connections 2023-08-24T10:09:18.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.102+0000 7fa0df11e640 1 -- 172.21.15.60:0/116492172 wait complete. 2023-08-24T10:09:18.114 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2023-08-24T10:09:17.545117+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:18.115 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:09:18.115 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:18.115 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:09:18.116 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:18.116 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:09:18.116 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:18.116 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:09:18.117 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:18.117 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:09:18.117 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:18.117 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:09:18.117 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:18.118 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:09:18.118 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:18.118 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:09:18.118 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:18.119 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:09:18.119 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:18.119 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:09:18.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.226+0000 7f2dd519e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:18.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.229+0000 7f2dd519e640 1 Processor -- start 2023-08-24T10:09:18.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.229+0000 7f2dd519e640 1 -- start start 2023-08-24T10:09:18.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.230+0000 7f2dd519e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2dd0132680 0x7f2dd0134aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:18.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.230+0000 7f2dd519e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd006dfc0 con 0x7f2dd0070c30 2023-08-24T10:09:18.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.230+0000 7f2dd519e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd006d9b0 con 0x7f2dd0132680 2023-08-24T10:09:18.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.230+0000 7f2dce59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2dd0132680 0x7f2dd0134aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:18.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.230+0000 7f2dce59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2dd0132680 0x7f2dd0134aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34088/0 (socket says 172.21.15.60:34088) 2023-08-24T10:09:18.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.230+0000 7f2dce59c640 1 -- 172.21.15.60:0/203478153 learned_addr learned my addr 172.21.15.60:0/203478153 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:18.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.230+0000 7f2dced9d640 1 --1- 172.21.15.60:0/203478153 >> v1:172.21.15.60:6789/0 conn(0x7f2dd0070c30 0x7f2dd0071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38776/0 (socket says 172.21.15.60:38776) 2023-08-24T10:09:18.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.230+0000 7f2dce59c640 1 -- 172.21.15.60:0/203478153 >> v1:172.21.15.60:6789/0 conn(0x7f2dd0070c30 legacy=0x7f2dd0071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:18.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.230+0000 7f2dce59c640 1 -- 172.21.15.60:0/203478153 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd006e6e0 con 0x7f2dd0132680 2023-08-24T10:09:18.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.230+0000 7f2dce59c640 1 --2- 172.21.15.60:0/203478153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2dd0132680 0x7f2dd0134aa0 crc :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b240aeb7a01d2b93 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:18.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.231+0000 7f2dcdd9b640 1 -- 172.21.15.60:0/203478153 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2db8009dd0 con 0x7f2dd0132680 2023-08-24T10:09:18.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.231+0000 7f2dcdd9b640 1 -- 172.21.15.60:0/203478153 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f2db800e230 con 0x7f2dd0132680 2023-08-24T10:09:18.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.231+0000 7f2dcdd9b640 1 -- 172.21.15.60:0/203478153 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2db800e4d0 con 0x7f2dd0132680 2023-08-24T10:09:18.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.231+0000 7f2dd519e640 1 -- 172.21.15.60:0/203478153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2dd0132680 msgr2=0x7f2dd0134aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:18.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.231+0000 7f2dd519e640 1 --2- 172.21.15.60:0/203478153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2dd0132680 0x7f2dd0134aa0 crc :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:18.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.231+0000 7f2dd519e640 1 -- 172.21.15.60:0/203478153 shutdown_connections 2023-08-24T10:09:18.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.231+0000 7f2dd519e640 1 --2- 172.21.15.60:0/203478153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2dd0132680 0x7f2dd0134aa0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:18.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.231+0000 7f2dd519e640 1 -- 172.21.15.60:0/203478153 >> 172.21.15.60:0/203478153 conn(0x7f2dd009e110 msgr2=0x7f2dd009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:18.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.231+0000 7f2dd519e640 1 -- 172.21.15.60:0/203478153 shutdown_connections 2023-08-24T10:09:18.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.231+0000 7f2dd519e640 1 -- 172.21.15.60:0/203478153 wait complete. 2023-08-24T10:09:18.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.232+0000 7f2dd519e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:18.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.233+0000 7f2dd519e640 1 Processor -- start 2023-08-24T10:09:18.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.233+0000 7f2dd519e640 1 -- start start 2023-08-24T10:09:18.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.233+0000 7f2dd519e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2dd0070c30 0x7f2dd012c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:18.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.233+0000 7f2dd519e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd012f4a0 con 0x7f2dd0070c30 2023-08-24T10:09:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.234+0000 7f2dced9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2dd0070c30 0x7f2dd012c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.234+0000 7f2dced9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2dd0070c30 0x7f2dd012c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34090/0 (socket says 172.21.15.60:34090) 2023-08-24T10:09:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.234+0000 7f2dced9d640 1 -- 172.21.15.60:0/49682975 learned_addr learned my addr 172.21.15.60:0/49682975 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:18.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.234+0000 7f2dced9d640 1 -- 172.21.15.60:0/49682975 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd01300b0 con 0x7f2dd0070c30 2023-08-24T10:09:18.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.234+0000 7f2dced9d640 1 --2- 172.21.15.60:0/49682975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2dd0070c30 0x7f2dd012c1a0 crc :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:18.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.234+0000 7f2dc77fe640 1 -- 172.21.15.60:0/49682975 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2dc80071e0 con 0x7f2dd0070c30 2023-08-24T10:09:18.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.234+0000 7f2dc77fe640 1 -- 172.21.15.60:0/49682975 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f2dc8007340 con 0x7f2dd0070c30 2023-08-24T10:09:18.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.234+0000 7f2dd519e640 1 -- 172.21.15.60:0/49682975 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd0098b50 con 0x7f2dd0070c30 2023-08-24T10:09:18.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.234+0000 7f2dc77fe640 1 -- 172.21.15.60:0/49682975 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2dc80075e0 con 0x7f2dd0070c30 2023-08-24T10:09:18.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.234+0000 7f2dd519e640 1 -- 172.21.15.60:0/49682975 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd012dd40 con 0x7f2dd0070c30 2023-08-24T10:09:18.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.235+0000 7f2dc77fe640 1 -- 172.21.15.60:0/49682975 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f2dc80057e0 con 0x7f2dd0070c30 2023-08-24T10:09:18.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.235+0000 7f2dd519e640 1 -- 172.21.15.60:0/49682975 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd0061800 con 0x7f2dd0070c30 2023-08-24T10:09:18.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.235+0000 7f2dc77fe640 1 --2- 172.21.15.60:0/49682975 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2db0037180 0x7f2db0039690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:18.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.235+0000 7f2dc77fe640 1 -- 172.21.15.60:0/49682975 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f2dc8046400 con 0x7f2dd0070c30 2023-08-24T10:09:18.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.236+0000 7f2dce59c640 1 --2- 172.21.15.60:0/49682975 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2db0037180 0x7f2db0039690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:18.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.236+0000 7f2dce59c640 1 --2- 172.21.15.60:0/49682975 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2db0037180 0x7f2db0039690 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.240+0000 7f2dc77fe640 1 -- 172.21.15.60:0/49682975 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f2dc800d330 con 0x7f2dd0070c30 2023-08-24T10:09:18.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.488+0000 7f2dd519e640 1 -- 172.21.15.60:0/49682975 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2dd00706e0 con 0x7f2dd0070c30 2023-08-24T10:09:18.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.488+0000 7f2dc77fe640 1 -- 172.21.15.60:0/49682975 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f2dc800d150 con 0x7f2dd0070c30 2023-08-24T10:09:18.489 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:18.490 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":269,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:09:18.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.490+0000 7f2dd519e640 1 -- 172.21.15.60:0/49682975 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2db0037180 msgr2=0x7f2db0039690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:18.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.490+0000 7f2dd519e640 1 --2- 172.21.15.60:0/49682975 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2db0037180 0x7f2db0039690 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:18.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.490+0000 7f2dd519e640 1 -- 172.21.15.60:0/49682975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2dd0070c30 msgr2=0x7f2dd012c1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:18.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.490+0000 7f2dd519e640 1 --2- 172.21.15.60:0/49682975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2dd0070c30 0x7f2dd012c1a0 crc :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:18.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.490+0000 7f2dd519e640 1 -- 172.21.15.60:0/49682975 shutdown_connections 2023-08-24T10:09:18.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.490+0000 7f2dd519e640 1 --2- 172.21.15.60:0/49682975 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2db0037180 0x7f2db0039690 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:18.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.490+0000 7f2dd519e640 1 --2- 172.21.15.60:0/49682975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2dd0070c30 0x7f2dd012c1a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:18.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.490+0000 7f2dd519e640 1 -- 172.21.15.60:0/49682975 >> 172.21.15.60:0/49682975 conn(0x7f2dd009e110 msgr2=0x7f2dd01b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:18.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.490+0000 7f2dd519e640 1 -- 172.21.15.60:0/49682975 shutdown_connections 2023-08-24T10:09:18.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.490+0000 7f2dd519e640 1 -- 172.21.15.60:0/49682975 wait complete. 2023-08-24T10:09:18.502 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:09:18.502 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:18.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.609+0000 7f1fa559e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:18.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.613+0000 7f1fa559e640 1 Processor -- start 2023-08-24T10:09:18.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.613+0000 7f1fa559e640 1 -- start start 2023-08-24T10:09:18.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.613+0000 7f1fa559e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1fa006d300 0x7f1fa012bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:18.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.613+0000 7f1fa559e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa0099440 con 0x7f1fa012c1c0 2023-08-24T10:09:18.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.613+0000 7f1fa559e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa009a050 con 0x7f1fa006d300 2023-08-24T10:09:18.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.613+0000 7f1f9effd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1fa006d300 0x7f1fa012bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:18.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.613+0000 7f1f9effd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1fa006d300 0x7f1fa012bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34104/0 (socket says 172.21.15.60:34104) 2023-08-24T10:09:18.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.613+0000 7f1f9e7fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1fa012c1c0 0x7f1fa012ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38784/0 (socket says 172.21.15.60:38784) 2023-08-24T10:09:18.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.613+0000 7f1f9effd640 1 -- 172.21.15.60:0/798574959 learned_addr learned my addr 172.21.15.60:0/798574959 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:18.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.614+0000 7f1f9effd640 1 -- 172.21.15.60:0/798574959 >> v1:172.21.15.60:6789/0 conn(0x7f1fa012c1c0 legacy=0x7f1fa012ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:18.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.614+0000 7f1f9effd640 1 -- 172.21.15.60:0/798574959 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa012b9d0 con 0x7f1fa006d300 2023-08-24T10:09:18.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.614+0000 7f1f9effd640 1 --2- 172.21.15.60:0/798574959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa006d300 0x7f1fa012bc80 crc :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe46d5b53fd97ded server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:18.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.614+0000 7f1f9dffb640 1 -- 172.21.15.60:0/798574959 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1f90009dd0 con 0x7f1fa006d300 2023-08-24T10:09:18.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.614+0000 7f1f9dffb640 1 -- 172.21.15.60:0/798574959 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1f9000e230 con 0x7f1fa006d300 2023-08-24T10:09:18.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.614+0000 7f1f9dffb640 1 -- 172.21.15.60:0/798574959 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1f9000e4d0 con 0x7f1fa006d300 2023-08-24T10:09:18.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.615+0000 7f1fa559e640 1 -- 172.21.15.60:0/798574959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa006d300 msgr2=0x7f1fa012bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:18.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.615+0000 7f1fa559e640 1 --2- 172.21.15.60:0/798574959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa006d300 0x7f1fa012bc80 crc :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:18.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.615+0000 7f1fa559e640 1 -- 172.21.15.60:0/798574959 shutdown_connections 2023-08-24T10:09:18.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.615+0000 7f1fa559e640 1 --2- 172.21.15.60:0/798574959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa006d300 0x7f1fa012bc80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:18.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.615+0000 7f1fa559e640 1 -- 172.21.15.60:0/798574959 >> 172.21.15.60:0/798574959 conn(0x7f1fa009e110 msgr2=0x7f1fa009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:18.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.615+0000 7f1fa559e640 1 -- 172.21.15.60:0/798574959 shutdown_connections 2023-08-24T10:09:18.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.615+0000 7f1fa559e640 1 -- 172.21.15.60:0/798574959 wait complete. 2023-08-24T10:09:18.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.615+0000 7f1fa559e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:18.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.616+0000 7f1fa559e640 1 Processor -- start 2023-08-24T10:09:18.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.616+0000 7f1fa559e640 1 -- start start 2023-08-24T10:09:18.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.617+0000 7f1fa559e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa006d300 0x7f1fa006ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:18.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.617+0000 7f1fa559e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa0071110 con 0x7f1fa006d300 2023-08-24T10:09:18.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.617+0000 7f1f9effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa006d300 0x7f1fa006ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:18.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.617+0000 7f1f9effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa006d300 0x7f1fa006ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34112/0 (socket says 172.21.15.60:34112) 2023-08-24T10:09:18.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.617+0000 7f1f9effd640 1 -- 172.21.15.60:0/2100370359 learned_addr learned my addr 172.21.15.60:0/2100370359 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:18.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.617+0000 7f1f9effd640 1 -- 172.21.15.60:0/2100370359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa0071340 con 0x7f1fa006d300 2023-08-24T10:09:18.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.617+0000 7f1f9effd640 1 --2- 172.21.15.60:0/2100370359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa006d300 0x7f1fa006ed60 crc :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:18.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.617+0000 7f1f877fe640 1 -- 172.21.15.60:0/2100370359 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1f90014540 con 0x7f1fa006d300 2023-08-24T10:09:18.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.617+0000 7f1f877fe640 1 -- 172.21.15.60:0/2100370359 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1f900146a0 con 0x7f1fa006d300 2023-08-24T10:09:18.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.617+0000 7f1f877fe640 1 -- 172.21.15.60:0/2100370359 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1f90014940 con 0x7f1fa006d300 2023-08-24T10:09:18.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.617+0000 7f1fa559e640 1 -- 172.21.15.60:0/2100370359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa00702d0 con 0x7f1fa006d300 2023-08-24T10:09:18.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.617+0000 7f1fa559e640 1 -- 172.21.15.60:0/2100370359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa0071a10 con 0x7f1fa006d300 2023-08-24T10:09:18.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.618+0000 7f1f877fe640 1 -- 172.21.15.60:0/2100370359 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f1f90014af0 con 0x7f1fa006d300 2023-08-24T10:09:18.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.618+0000 7f1fa559e640 1 -- 172.21.15.60:0/2100370359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa0061800 con 0x7f1fa006d300 2023-08-24T10:09:18.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.619+0000 7f1f877fe640 1 --2- 172.21.15.60:0/2100370359 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1f8c037130 0x7f1f8c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:18.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.619+0000 7f1f877fe640 1 -- 172.21.15.60:0/2100370359 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f1f90019040 con 0x7f1fa006d300 2023-08-24T10:09:18.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.619+0000 7f1f9e7fc640 1 --2- 172.21.15.60:0/2100370359 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1f8c037130 0x7f1f8c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:18.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.619+0000 7f1f9e7fc640 1 --2- 172.21.15.60:0/2100370359 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1f8c037130 0x7f1f8c039640 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:18.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.623+0000 7f1f877fe640 1 -- 172.21.15.60:0/2100370359 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f1f90018190 con 0x7f1fa006d300 2023-08-24T10:09:18.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.799+0000 7f1fa559e640 1 -- 172.21.15.60:0/2100370359 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1fa01308e0 con 0x7f1f8c037130 2023-08-24T10:09:18.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.800+0000 7f1f877fe640 1 -- 172.21.15.60:0/2100370359 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f1fa01308e0 con 0x7f1f8c037130 2023-08-24T10:09:18.801 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:18.802 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:18.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.802+0000 7f1fa559e640 1 -- 172.21.15.60:0/2100370359 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1f8c037130 msgr2=0x7f1f8c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:18.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.802+0000 7f1fa559e640 1 --2- 172.21.15.60:0/2100370359 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1f8c037130 0x7f1f8c039640 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:18.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.802+0000 7f1fa559e640 1 -- 172.21.15.60:0/2100370359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa006d300 msgr2=0x7f1fa006ed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:18.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.802+0000 7f1fa559e640 1 --2- 172.21.15.60:0/2100370359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa006d300 0x7f1fa006ed60 crc :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:18.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.802+0000 7f1fa559e640 1 -- 172.21.15.60:0/2100370359 shutdown_connections 2023-08-24T10:09:18.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.802+0000 7f1fa559e640 1 --2- 172.21.15.60:0/2100370359 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1f8c037130 0x7f1f8c039640 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:18.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.802+0000 7f1fa559e640 1 --2- 172.21.15.60:0/2100370359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa006d300 0x7f1fa006ed60 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:18.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.802+0000 7f1fa559e640 1 -- 172.21.15.60:0/2100370359 >> 172.21.15.60:0/2100370359 conn(0x7f1fa009e110 msgr2=0x7f1fa009bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:18.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.802+0000 7f1fa559e640 1 -- 172.21.15.60:0/2100370359 shutdown_connections 2023-08-24T10:09:18.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:18.802+0000 7f1fa559e640 1 -- 172.21.15.60:0/2100370359 wait complete. 2023-08-24T10:09:18.813 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2023-08-24T10:09:17.545117+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:21.816 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:21.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.936+0000 7f5d874a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:21.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.940+0000 7f5d874a3640 1 Processor -- start 2023-08-24T10:09:21.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.940+0000 7f5d874a3640 1 -- start start 2023-08-24T10:09:21.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.941+0000 7f5d874a3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5d80131d40 0x7f5d80134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:21.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.941+0000 7f5d874a3640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d80099440 con 0x7f5d8006d300 2023-08-24T10:09:21.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.941+0000 7f5d874a3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8009a050 con 0x7f5d80131d40 2023-08-24T10:09:21.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.941+0000 7f5d8523f640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5d8006d300 0x7f5d8012bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38788/0 (socket says 172.21.15.60:38788) 2023-08-24T10:09:21.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.941+0000 7f5d8523f640 1 -- 172.21.15.60:0/1911106520 learned_addr learned my addr 172.21.15.60:0/1911106520 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:09:21.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.941+0000 7f5d84a3e640 1 --2- 172.21.15.60:0/1911106520 >> v2:172.21.15.60:3300/0 conn(0x7f5d80131d40 0x7f5d80134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:21.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.941+0000 7f5d77fff640 1 -- 172.21.15.60:0/1911106520 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1590191862 0 0) 0x7f5d80099440 con 0x7f5d8006d300 2023-08-24T10:09:21.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.941+0000 7f5d77fff640 1 -- 172.21.15.60:0/1911106520 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d6c005700 con 0x7f5d8006d300 2023-08-24T10:09:21.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.941+0000 7f5d84a3e640 1 -- 172.21.15.60:0/1911106520 >> v1:172.21.15.60:6789/0 conn(0x7f5d8006d300 legacy=0x7f5d8012bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:21.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.941+0000 7f5d84a3e640 1 -- 172.21.15.60:0/1911106520 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8012b9d0 con 0x7f5d80131d40 2023-08-24T10:09:21.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.941+0000 7f5d84a3e640 1 --2- 172.21.15.60:0/1911106520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d80131d40 0x7f5d80134160 crc :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc019c054b510a31 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:21.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.941+0000 7f5d77fff640 1 -- 172.21.15.60:0/1911106520 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5d70009cd0 con 0x7f5d80131d40 2023-08-24T10:09:21.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.941+0000 7f5d77fff640 1 -- 172.21.15.60:0/1911106520 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5d70009e30 con 0x7f5d80131d40 2023-08-24T10:09:21.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.941+0000 7f5d77fff640 1 -- 172.21.15.60:0/1911106520 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5d7000e360 con 0x7f5d80131d40 2023-08-24T10:09:21.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.942+0000 7f5d874a3640 1 -- 172.21.15.60:0/1911106520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d80131d40 msgr2=0x7f5d80134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:21.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.942+0000 7f5d874a3640 1 --2- 172.21.15.60:0/1911106520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d80131d40 0x7f5d80134160 crc :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:21.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.942+0000 7f5d874a3640 1 -- 172.21.15.60:0/1911106520 shutdown_connections 2023-08-24T10:09:21.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.942+0000 7f5d874a3640 1 --2- 172.21.15.60:0/1911106520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d80131d40 0x7f5d80134160 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:21.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.942+0000 7f5d874a3640 1 -- 172.21.15.60:0/1911106520 >> 172.21.15.60:0/1911106520 conn(0x7f5d8009e110 msgr2=0x7f5d8009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:21.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.942+0000 7f5d874a3640 1 -- 172.21.15.60:0/1911106520 shutdown_connections 2023-08-24T10:09:21.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.942+0000 7f5d874a3640 1 -- 172.21.15.60:0/1911106520 wait complete. 2023-08-24T10:09:21.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.942+0000 7f5d874a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:21.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.943+0000 7f5d874a3640 1 Processor -- start 2023-08-24T10:09:21.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.943+0000 7f5d874a3640 1 -- start start 2023-08-24T10:09:21.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.944+0000 7f5d874a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d8006d300 0x7f5d80126450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:21.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.944+0000 7f5d874a3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8006ee10 con 0x7f5d8006d300 2023-08-24T10:09:21.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.944+0000 7f5d8523f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d8006d300 0x7f5d80126450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:21.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.944+0000 7f5d8523f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d8006d300 0x7f5d80126450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34136/0 (socket says 172.21.15.60:34136) 2023-08-24T10:09:21.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.944+0000 7f5d8523f640 1 -- 172.21.15.60:0/81654609 learned_addr learned my addr 172.21.15.60:0/81654609 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:21.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.944+0000 7f5d8523f640 1 -- 172.21.15.60:0/81654609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8006df20 con 0x7f5d8006d300 2023-08-24T10:09:21.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.944+0000 7f5d8523f640 1 --2- 172.21.15.60:0/81654609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d8006d300 0x7f5d80126450 crc :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:21.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.945+0000 7f5d75ffb640 1 -- 172.21.15.60:0/81654609 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5d7c0059d0 con 0x7f5d8006d300 2023-08-24T10:09:21.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.945+0000 7f5d75ffb640 1 -- 172.21.15.60:0/81654609 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5d7c005b30 con 0x7f5d8006d300 2023-08-24T10:09:21.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.945+0000 7f5d874a3640 1 -- 172.21.15.60:0/81654609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8006e1e0 con 0x7f5d8006d300 2023-08-24T10:09:21.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.945+0000 7f5d75ffb640 1 -- 172.21.15.60:0/81654609 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5d7c005dd0 con 0x7f5d8006d300 2023-08-24T10:09:21.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.945+0000 7f5d874a3640 1 -- 172.21.15.60:0/81654609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d80071be0 con 0x7f5d8006d300 2023-08-24T10:09:21.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.946+0000 7f5d75ffb640 1 -- 172.21.15.60:0/81654609 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f5d7c0070b0 con 0x7f5d8006d300 2023-08-24T10:09:21.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.946+0000 7f5d75ffb640 1 --2- 172.21.15.60:0/81654609 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5d64037130 0x7f5d64039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:21.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.946+0000 7f5d75ffb640 1 -- 172.21.15.60:0/81654609 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f5d7c046250 con 0x7f5d8006d300 2023-08-24T10:09:21.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.946+0000 7f5d874a3640 1 -- 172.21.15.60:0/81654609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d801318e0 con 0x7f5d8006d300 2023-08-24T10:09:21.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.946+0000 7f5d84a3e640 1 --2- 172.21.15.60:0/81654609 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5d64037130 0x7f5d64039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:21.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.947+0000 7f5d84a3e640 1 --2- 172.21.15.60:0/81654609 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5d64037130 0x7f5d64039640 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:21.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:21.951+0000 7f5d75ffb640 1 -- 172.21.15.60:0/81654609 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f5d7c00d6e0 con 0x7f5d8006d300 2023-08-24T10:09:22.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.123+0000 7f5d874a3640 1 -- 172.21.15.60:0/81654609 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5d80097710 con 0x7f5d64037130 2023-08-24T10:09:22.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.124+0000 7f5d75ffb640 1 -- 172.21.15.60:0/81654609 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f5d70009300 con 0x7f5d64037130 2023-08-24T10:09:22.125 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:22.126 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:22.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.126+0000 7f5d874a3640 1 -- 172.21.15.60:0/81654609 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5d64037130 msgr2=0x7f5d64039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:22.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.126+0000 7f5d874a3640 1 --2- 172.21.15.60:0/81654609 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5d64037130 0x7f5d64039640 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:22.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.126+0000 7f5d874a3640 1 -- 172.21.15.60:0/81654609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d8006d300 msgr2=0x7f5d80126450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:22.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.126+0000 7f5d874a3640 1 --2- 172.21.15.60:0/81654609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d8006d300 0x7f5d80126450 crc :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:22.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.126+0000 7f5d874a3640 1 -- 172.21.15.60:0/81654609 shutdown_connections 2023-08-24T10:09:22.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.126+0000 7f5d874a3640 1 --2- 172.21.15.60:0/81654609 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5d64037130 0x7f5d64039640 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:22.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.126+0000 7f5d874a3640 1 --2- 172.21.15.60:0/81654609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d8006d300 0x7f5d80126450 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:22.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.126+0000 7f5d874a3640 1 -- 172.21.15.60:0/81654609 >> 172.21.15.60:0/81654609 conn(0x7f5d8009e110 msgr2=0x7f5d800964f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:22.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.126+0000 7f5d874a3640 1 -- 172.21.15.60:0/81654609 shutdown_connections 2023-08-24T10:09:22.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.126+0000 7f5d874a3640 1 -- 172.21.15.60:0/81654609 wait complete. 2023-08-24T10:09:22.137 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2023-08-24T10:09:21.545883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:22.138 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:22.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.245+0000 7fd45059e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:22.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.249+0000 7fd45059e640 1 Processor -- start 2023-08-24T10:09:22.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.249+0000 7fd45059e640 1 -- start start 2023-08-24T10:09:22.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.250+0000 7fd45059e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd448070110 0x7fd4481356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:22.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.250+0000 7fd45059e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd448096f10 con 0x7fd448071e40 2023-08-24T10:09:22.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.250+0000 7fd45059e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4480977f0 con 0x7fd448070110 2023-08-24T10:09:22.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.250+0000 7fd44db39640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd448070110 0x7fd4481356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:22.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.250+0000 7fd44db39640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd448070110 0x7fd4481356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34146/0 (socket says 172.21.15.60:34146) 2023-08-24T10:09:22.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.250+0000 7fd44e33a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd448071e40 0x7fd448096800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38798/0 (socket says 172.21.15.60:38798) 2023-08-24T10:09:22.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.250+0000 7fd44db39640 1 -- 172.21.15.60:0/1821806712 learned_addr learned my addr 172.21.15.60:0/1821806712 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:22.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.250+0000 7fd44db39640 1 -- 172.21.15.60:0/1821806712 >> v1:172.21.15.60:6789/0 conn(0x7fd448071e40 legacy=0x7fd448096800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:22.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.250+0000 7fd44db39640 1 -- 172.21.15.60:0/1821806712 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44809a280 con 0x7fd448070110 2023-08-24T10:09:22.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.250+0000 7fd44db39640 1 --2- 172.21.15.60:0/1821806712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd448070110 0x7fd4481356a0 crc :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53478fa7b13398f0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:22.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.250+0000 7fd44d338640 1 -- 172.21.15.60:0/1821806712 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd438009dd0 con 0x7fd448070110 2023-08-24T10:09:22.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.250+0000 7fd44d338640 1 -- 172.21.15.60:0/1821806712 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd43800e230 con 0x7fd448070110 2023-08-24T10:09:22.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.250+0000 7fd44d338640 1 -- 172.21.15.60:0/1821806712 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd43800e4d0 con 0x7fd448070110 2023-08-24T10:09:22.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.251+0000 7fd45059e640 1 -- 172.21.15.60:0/1821806712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd448070110 msgr2=0x7fd4481356a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:22.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.251+0000 7fd45059e640 1 --2- 172.21.15.60:0/1821806712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd448070110 0x7fd4481356a0 crc :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:22.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.251+0000 7fd45059e640 1 -- 172.21.15.60:0/1821806712 shutdown_connections 2023-08-24T10:09:22.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.251+0000 7fd45059e640 1 --2- 172.21.15.60:0/1821806712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd448070110 0x7fd4481356a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:22.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.251+0000 7fd45059e640 1 -- 172.21.15.60:0/1821806712 >> 172.21.15.60:0/1821806712 conn(0x7fd44809e110 msgr2=0x7fd44809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:22.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.251+0000 7fd45059e640 1 -- 172.21.15.60:0/1821806712 shutdown_connections 2023-08-24T10:09:22.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.251+0000 7fd45059e640 1 -- 172.21.15.60:0/1821806712 wait complete. 2023-08-24T10:09:22.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.251+0000 7fd45059e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:22.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.252+0000 7fd45059e640 1 Processor -- start 2023-08-24T10:09:22.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.252+0000 7fd45059e640 1 -- start start 2023-08-24T10:09:22.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.253+0000 7fd45059e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd448070110 0x7fd448093460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:22.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.253+0000 7fd45059e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44808ff10 con 0x7fd448070110 2023-08-24T10:09:22.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.253+0000 7fd44e33a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd448070110 0x7fd448093460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:22.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.253+0000 7fd44e33a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd448070110 0x7fd448093460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34156/0 (socket says 172.21.15.60:34156) 2023-08-24T10:09:22.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.253+0000 7fd44e33a640 1 -- 172.21.15.60:0/2189058068 learned_addr learned my addr 172.21.15.60:0/2189058068 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:22.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.253+0000 7fd44e33a640 1 -- 172.21.15.60:0/2189058068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd448090680 con 0x7fd448070110 2023-08-24T10:09:22.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.254+0000 7fd44e33a640 1 --2- 172.21.15.60:0/2189058068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd448070110 0x7fd448093460 crc :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:22.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.254+0000 7fd43effd640 1 -- 172.21.15.60:0/2189058068 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd4440071e0 con 0x7fd448070110 2023-08-24T10:09:22.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.254+0000 7fd43effd640 1 -- 172.21.15.60:0/2189058068 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd444007340 con 0x7fd448070110 2023-08-24T10:09:22.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.254+0000 7fd45059e640 1 -- 172.21.15.60:0/2189058068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd448092e80 con 0x7fd448070110 2023-08-24T10:09:22.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.254+0000 7fd43effd640 1 -- 172.21.15.60:0/2189058068 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd4440075e0 con 0x7fd448070110 2023-08-24T10:09:22.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.254+0000 7fd45059e640 1 -- 172.21.15.60:0/2189058068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd448092b70 con 0x7fd448070110 2023-08-24T10:09:22.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.255+0000 7fd43effd640 1 -- 172.21.15.60:0/2189058068 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fd4440057e0 con 0x7fd448070110 2023-08-24T10:09:22.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.255+0000 7fd45059e640 1 -- 172.21.15.60:0/2189058068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd448061800 con 0x7fd448070110 2023-08-24T10:09:22.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.255+0000 7fd43effd640 1 --2- 172.21.15.60:0/2189058068 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd42c037130 0x7fd42c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:22.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.255+0000 7fd43effd640 1 -- 172.21.15.60:0/2189058068 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fd444046330 con 0x7fd448070110 2023-08-24T10:09:22.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.255+0000 7fd44db39640 1 --2- 172.21.15.60:0/2189058068 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd42c037130 0x7fd42c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:22.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.256+0000 7fd44db39640 1 --2- 172.21.15.60:0/2189058068 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd42c037130 0x7fd42c039640 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:22.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.259+0000 7fd43effd640 1 -- 172.21.15.60:0/2189058068 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fd44400d330 con 0x7fd448070110 2023-08-24T10:09:22.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.433+0000 7fd45059e640 1 -- 172.21.15.60:0/2189058068 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd4480932f0 con 0x7fd42c037130 2023-08-24T10:09:22.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.434+0000 7fd43effd640 1 -- 172.21.15.60:0/2189058068 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7fd4480932f0 con 0x7fd42c037130 2023-08-24T10:09:22.436 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:22.438 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:22.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.436+0000 7fd45059e640 1 -- 172.21.15.60:0/2189058068 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd42c037130 msgr2=0x7fd42c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:22.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.436+0000 7fd45059e640 1 --2- 172.21.15.60:0/2189058068 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd42c037130 0x7fd42c039640 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:22.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.436+0000 7fd45059e640 1 -- 172.21.15.60:0/2189058068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd448070110 msgr2=0x7fd448093460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:22.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.436+0000 7fd45059e640 1 --2- 172.21.15.60:0/2189058068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd448070110 0x7fd448093460 crc :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:22.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.436+0000 7fd45059e640 1 -- 172.21.15.60:0/2189058068 shutdown_connections 2023-08-24T10:09:22.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.436+0000 7fd45059e640 1 --2- 172.21.15.60:0/2189058068 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fd42c037130 0x7fd42c039640 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:22.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.436+0000 7fd45059e640 1 --2- 172.21.15.60:0/2189058068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd448070110 0x7fd448093460 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:22.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.436+0000 7fd45059e640 1 -- 172.21.15.60:0/2189058068 >> 172.21.15.60:0/2189058068 conn(0x7fd44809e110 msgr2=0x7fd448098490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:22.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.436+0000 7fd45059e640 1 -- 172.21.15.60:0/2189058068 shutdown_connections 2023-08-24T10:09:22.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.436+0000 7fd45059e640 1 -- 172.21.15.60:0/2189058068 wait complete. 2023-08-24T10:09:22.442 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~255s 2023-08-24T10:09:22.442 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~255s 2023-08-24T10:09:22.443 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~255s 2023-08-24T10:09:22.443 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~255s 2023-08-24T10:09:22.448 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2023-08-24T10:09:21.545883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:22.449 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:09:22.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:22.450 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:09:22.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:22.450 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:09:22.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:22.451 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:09:22.451 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:22.451 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:09:22.451 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:22.452 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:09:22.452 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:22.452 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:09:22.452 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:22.453 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:09:22.453 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:22.453 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:09:22.453 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:22.454 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:09:22.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.561+0000 7fcf06b1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:22.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.564+0000 7fcf06b1e640 1 Processor -- start 2023-08-24T10:09:22.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.564+0000 7fcf06b1e640 1 -- start start 2023-08-24T10:09:22.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.565+0000 7fcf06b1e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcf00131d20 0x7fcf00134140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:22.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.565+0000 7fcf06b1e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf0009a650 con 0x7fcf0012ece0 2023-08-24T10:09:22.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.565+0000 7fcf06b1e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf0006da00 con 0x7fcf00131d20 2023-08-24T10:09:22.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.565+0000 7fcefffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcf00131d20 0x7fcf00134140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:22.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.565+0000 7fcefffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcf00131d20 0x7fcf00134140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34158/0 (socket says 172.21.15.60:34158) 2023-08-24T10:09:22.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.565+0000 7fcefffff640 1 -- 172.21.15.60:0/692641822 learned_addr learned my addr 172.21.15.60:0/692641822 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:22.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.565+0000 7fcf048ba640 1 --1- 172.21.15.60:0/692641822 >> v1:172.21.15.60:6789/0 conn(0x7fcf0012ece0 0x7fcf0012f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38806/0 (socket says 172.21.15.60:38806) 2023-08-24T10:09:22.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.565+0000 7fcefffff640 1 -- 172.21.15.60:0/692641822 >> v1:172.21.15.60:6789/0 conn(0x7fcf0012ece0 legacy=0x7fcf0012f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:22.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.565+0000 7fcefffff640 1 -- 172.21.15.60:0/692641822 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0012ea10 con 0x7fcf00131d20 2023-08-24T10:09:22.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.565+0000 7fcefffff640 1 --2- 172.21.15.60:0/692641822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf00131d20 0x7fcf00134140 crc :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a941a891863fe95d server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:22.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.565+0000 7fceff7fe640 1 -- 172.21.15.60:0/692641822 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcef0009dd0 con 0x7fcf00131d20 2023-08-24T10:09:22.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.566+0000 7fceff7fe640 1 -- 172.21.15.60:0/692641822 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fcef000e230 con 0x7fcf00131d20 2023-08-24T10:09:22.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.566+0000 7fceff7fe640 1 -- 172.21.15.60:0/692641822 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcef000e4d0 con 0x7fcf00131d20 2023-08-24T10:09:22.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.566+0000 7fcf06b1e640 1 -- 172.21.15.60:0/692641822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf00131d20 msgr2=0x7fcf00134140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:22.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.566+0000 7fcf06b1e640 1 --2- 172.21.15.60:0/692641822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf00131d20 0x7fcf00134140 crc :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:22.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.566+0000 7fcf06b1e640 1 -- 172.21.15.60:0/692641822 shutdown_connections 2023-08-24T10:09:22.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.566+0000 7fcf06b1e640 1 --2- 172.21.15.60:0/692641822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf00131d20 0x7fcf00134140 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:22.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.566+0000 7fcf06b1e640 1 -- 172.21.15.60:0/692641822 >> 172.21.15.60:0/692641822 conn(0x7fcf0009e110 msgr2=0x7fcf0009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:22.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.566+0000 7fcf06b1e640 1 -- 172.21.15.60:0/692641822 shutdown_connections 2023-08-24T10:09:22.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.566+0000 7fcf06b1e640 1 -- 172.21.15.60:0/692641822 wait complete. 2023-08-24T10:09:22.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.567+0000 7fcf06b1e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:22.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.567+0000 7fcf06b1e640 1 Processor -- start 2023-08-24T10:09:22.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.567+0000 7fcf06b1e640 1 -- start start 2023-08-24T10:09:22.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.568+0000 7fcf06b1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf0012ece0 0x7fcf001c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:22.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.568+0000 7fcf06b1e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0012bca0 con 0x7fcf0012ece0 2023-08-24T10:09:22.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.568+0000 7fcf048ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf0012ece0 0x7fcf001c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.568+0000 7fcf048ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf0012ece0 0x7fcf001c7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34164/0 (socket says 172.21.15.60:34164) 2023-08-24T10:09:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.568+0000 7fcf048ba640 1 -- 172.21.15.60:0/2743661519 learned_addr learned my addr 172.21.15.60:0/2743661519 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:22.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.569+0000 7fcf048ba640 1 -- 172.21.15.60:0/2743661519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf00071bb0 con 0x7fcf0012ece0 2023-08-24T10:09:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.569+0000 7fcf048ba640 1 --2- 172.21.15.60:0/2743661519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf0012ece0 0x7fcf001c7490 crc :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.569+0000 7fcefd7fa640 1 -- 172.21.15.60:0/2743661519 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcee80061e0 con 0x7fcf0012ece0 2023-08-24T10:09:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.569+0000 7fcefd7fa640 1 -- 172.21.15.60:0/2743661519 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fcee8006340 con 0x7fcf0012ece0 2023-08-24T10:09:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.569+0000 7fcf06b1e640 1 -- 172.21.15.60:0/2743661519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0012b370 con 0x7fcf0012ece0 2023-08-24T10:09:22.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.569+0000 7fcefd7fa640 1 -- 172.21.15.60:0/2743661519 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcee80065e0 con 0x7fcf0012ece0 2023-08-24T10:09:22.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.569+0000 7fcf06b1e640 1 -- 172.21.15.60:0/2743661519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf00129bd0 con 0x7fcf0012ece0 2023-08-24T10:09:22.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.570+0000 7fcefd7fa640 1 -- 172.21.15.60:0/2743661519 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fcee8004790 con 0x7fcf0012ece0 2023-08-24T10:09:22.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.570+0000 7fcf06b1e640 1 -- 172.21.15.60:0/2743661519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf0012ba10 con 0x7fcf0012ece0 2023-08-24T10:09:22.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.570+0000 7fcefd7fa640 1 --2- 172.21.15.60:0/2743661519 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcee00370e0 0x7fcee00395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:22.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.570+0000 7fcefd7fa640 1 -- 172.21.15.60:0/2743661519 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fcee8046350 con 0x7fcf0012ece0 2023-08-24T10:09:22.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.570+0000 7fcefffff640 1 --2- 172.21.15.60:0/2743661519 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcee00370e0 0x7fcee00395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:22.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.571+0000 7fcefffff640 1 --2- 172.21.15.60:0/2743661519 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcee00370e0 0x7fcee00395f0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:22.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.575+0000 7fcefd7fa640 1 -- 172.21.15.60:0/2743661519 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fcee800d330 con 0x7fcf0012ece0 2023-08-24T10:09:22.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.821+0000 7fcf06b1e640 1 -- 172.21.15.60:0/2743661519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fcf00061800 con 0x7fcf0012ece0 2023-08-24T10:09:22.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.821+0000 7fcefd7fa640 1 -- 172.21.15.60:0/2743661519 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7fcee800d150 con 0x7fcf0012ece0 2023-08-24T10:09:22.823 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:22.823 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":273,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:09:22.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.823+0000 7fcf06b1e640 1 -- 172.21.15.60:0/2743661519 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcee00370e0 msgr2=0x7fcee00395f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:22.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.823+0000 7fcf06b1e640 1 --2- 172.21.15.60:0/2743661519 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcee00370e0 0x7fcee00395f0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:22.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.823+0000 7fcf06b1e640 1 -- 172.21.15.60:0/2743661519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf0012ece0 msgr2=0x7fcf001c7490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:22.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.823+0000 7fcf06b1e640 1 --2- 172.21.15.60:0/2743661519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf0012ece0 0x7fcf001c7490 crc :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:22.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.823+0000 7fcf06b1e640 1 -- 172.21.15.60:0/2743661519 shutdown_connections 2023-08-24T10:09:22.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.823+0000 7fcf06b1e640 1 --2- 172.21.15.60:0/2743661519 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fcee00370e0 0x7fcee00395f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:22.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.823+0000 7fcf06b1e640 1 --2- 172.21.15.60:0/2743661519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf0012ece0 0x7fcf001c7490 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:22.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.823+0000 7fcf06b1e640 1 -- 172.21.15.60:0/2743661519 >> 172.21.15.60:0/2743661519 conn(0x7fcf0009e110 msgr2=0x7fcf00096820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:22.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.823+0000 7fcf06b1e640 1 -- 172.21.15.60:0/2743661519 shutdown_connections 2023-08-24T10:09:22.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.823+0000 7fcf06b1e640 1 -- 172.21.15.60:0/2743661519 wait complete. 2023-08-24T10:09:22.834 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:09:22.835 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:22.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.941+0000 7f7b0815e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:22.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.945+0000 7f7b0815e640 1 Processor -- start 2023-08-24T10:09:22.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.945+0000 7f7b0815e640 1 -- start start 2023-08-24T10:09:22.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.946+0000 7f7b0815e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7b0006d300 0x7f7b0012bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:22.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.946+0000 7f7b0815e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b00099440 con 0x7f7b0012c1c0 2023-08-24T10:09:22.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.946+0000 7f7b0815e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0009a050 con 0x7f7b0006d300 2023-08-24T10:09:22.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.946+0000 7f7b05efa640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7b0006d300 0x7f7b0012bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:22.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.946+0000 7f7b05efa640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7b0006d300 0x7f7b0012bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34166/0 (socket says 172.21.15.60:34166) 2023-08-24T10:09:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.946+0000 7f7b05efa640 1 -- 172.21.15.60:0/822614686 learned_addr learned my addr 172.21.15.60:0/822614686 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.946+0000 7f7b056f9640 1 --1- 172.21.15.60:0/822614686 >> v1:172.21.15.60:6789/0 conn(0x7f7b0012c1c0 0x7f7b0012ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38810/0 (socket says 172.21.15.60:38810) 2023-08-24T10:09:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.946+0000 7f7b05efa640 1 -- 172.21.15.60:0/822614686 >> v1:172.21.15.60:6789/0 conn(0x7f7b0012c1c0 legacy=0x7f7b0012ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.946+0000 7f7b05efa640 1 -- 172.21.15.60:0/822614686 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0012b9d0 con 0x7f7b0006d300 2023-08-24T10:09:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.947+0000 7f7b05efa640 1 --2- 172.21.15.60:0/822614686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0006d300 0x7f7b0012bc80 crc :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ad403529f0ff77 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.947+0000 7f7b04ef8640 1 -- 172.21.15.60:0/822614686 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7af0010040 con 0x7f7b0006d300 2023-08-24T10:09:22.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.947+0000 7f7b04ef8640 1 -- 172.21.15.60:0/822614686 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7af000b1e0 con 0x7f7b0006d300 2023-08-24T10:09:22.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.947+0000 7f7b04ef8640 1 -- 172.21.15.60:0/822614686 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7af000b480 con 0x7f7b0006d300 2023-08-24T10:09:22.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.947+0000 7f7b0815e640 1 -- 172.21.15.60:0/822614686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0006d300 msgr2=0x7f7b0012bc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:22.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.947+0000 7f7b0815e640 1 --2- 172.21.15.60:0/822614686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0006d300 0x7f7b0012bc80 crc :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:22.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.947+0000 7f7b0815e640 1 -- 172.21.15.60:0/822614686 shutdown_connections 2023-08-24T10:09:22.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.947+0000 7f7b0815e640 1 --2- 172.21.15.60:0/822614686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0006d300 0x7f7b0012bc80 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:22.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.947+0000 7f7b0815e640 1 -- 172.21.15.60:0/822614686 >> 172.21.15.60:0/822614686 conn(0x7f7b0009e110 msgr2=0x7f7b0009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:22.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.947+0000 7f7b0815e640 1 -- 172.21.15.60:0/822614686 shutdown_connections 2023-08-24T10:09:22.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.947+0000 7f7b0815e640 1 -- 172.21.15.60:0/822614686 wait complete. 2023-08-24T10:09:22.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.948+0000 7f7b0815e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:22.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.949+0000 7f7b0815e640 1 Processor -- start 2023-08-24T10:09:22.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.949+0000 7f7b0815e640 1 -- start start 2023-08-24T10:09:22.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.949+0000 7f7b0815e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0006d300 0x7f7b0006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:22.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.949+0000 7f7b0815e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0008d710 con 0x7f7b0006d300 2023-08-24T10:09:22.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.949+0000 7f7b05efa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0006d300 0x7f7b0006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:22.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.949+0000 7f7b05efa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0006d300 0x7f7b0006ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34180/0 (socket says 172.21.15.60:34180) 2023-08-24T10:09:22.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.949+0000 7f7b05efa640 1 -- 172.21.15.60:0/2366077664 learned_addr learned my addr 172.21.15.60:0/2366077664 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:22.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.950+0000 7f7b05efa640 1 -- 172.21.15.60:0/2366077664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0006ea80 con 0x7f7b0006d300 2023-08-24T10:09:22.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.950+0000 7f7b05efa640 1 --2- 172.21.15.60:0/2366077664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0006d300 0x7f7b0006ee40 crc :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:22.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.950+0000 7f7afe7fc640 1 -- 172.21.15.60:0/2366077664 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7af0010040 con 0x7f7b0006d300 2023-08-24T10:09:22.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.950+0000 7f7afe7fc640 1 -- 172.21.15.60:0/2366077664 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7af0008570 con 0x7f7b0006d300 2023-08-24T10:09:22.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.950+0000 7f7b0815e640 1 -- 172.21.15.60:0/2366077664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0006e770 con 0x7f7b0006d300 2023-08-24T10:09:22.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.950+0000 7f7afe7fc640 1 -- 172.21.15.60:0/2366077664 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7af0008810 con 0x7f7b0006d300 2023-08-24T10:09:22.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.950+0000 7f7b0815e640 1 -- 172.21.15.60:0/2366077664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b00070f60 con 0x7f7b0006d300 2023-08-24T10:09:22.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.951+0000 7f7b0815e640 1 -- 172.21.15.60:0/2366077664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b00061800 con 0x7f7b0006d300 2023-08-24T10:09:22.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.951+0000 7f7afe7fc640 1 -- 172.21.15.60:0/2366077664 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f7af0013040 con 0x7f7b0006d300 2023-08-24T10:09:22.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.951+0000 7f7afe7fc640 1 --2- 172.21.15.60:0/2366077664 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7aec037130 0x7f7aec039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:22.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.951+0000 7f7afe7fc640 1 -- 172.21.15.60:0/2366077664 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f7af0047cd0 con 0x7f7b0006d300 2023-08-24T10:09:22.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.951+0000 7f7b056f9640 1 --2- 172.21.15.60:0/2366077664 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7aec037130 0x7f7aec039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:22.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.952+0000 7f7b056f9640 1 --2- 172.21.15.60:0/2366077664 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7aec037130 0x7f7aec039640 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:22.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:22.956+0000 7f7afe7fc640 1 -- 172.21.15.60:0/2366077664 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f7af0007760 con 0x7f7b0006d300 2023-08-24T10:09:23.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:23.130+0000 7f7b0815e640 1 -- 172.21.15.60:0/2366077664 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7b0006da20 con 0x7f7aec037130 2023-08-24T10:09:23.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:23.132+0000 7f7afe7fc640 1 -- 172.21.15.60:0/2366077664 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f7af40070b0 con 0x7f7aec037130 2023-08-24T10:09:23.133 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:23.134 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:23.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:23.134+0000 7f7b0815e640 1 -- 172.21.15.60:0/2366077664 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7aec037130 msgr2=0x7f7aec039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:23.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:23.134+0000 7f7b0815e640 1 --2- 172.21.15.60:0/2366077664 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7aec037130 0x7f7aec039640 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:23.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:23.134+0000 7f7b0815e640 1 -- 172.21.15.60:0/2366077664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0006d300 msgr2=0x7f7b0006ee40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:23.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:23.134+0000 7f7b0815e640 1 --2- 172.21.15.60:0/2366077664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0006d300 0x7f7b0006ee40 crc :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:23.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:23.134+0000 7f7b0815e640 1 -- 172.21.15.60:0/2366077664 shutdown_connections 2023-08-24T10:09:23.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:23.134+0000 7f7b0815e640 1 --2- 172.21.15.60:0/2366077664 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7aec037130 0x7f7aec039640 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:23.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:23.134+0000 7f7b0815e640 1 --2- 172.21.15.60:0/2366077664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0006d300 0x7f7b0006ee40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:23.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:23.134+0000 7f7b0815e640 1 -- 172.21.15.60:0/2366077664 >> 172.21.15.60:0/2366077664 conn(0x7f7b0009e110 msgr2=0x7f7b0009bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:23.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:23.134+0000 7f7b0815e640 1 -- 172.21.15.60:0/2366077664 shutdown_connections 2023-08-24T10:09:23.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:23.134+0000 7f7b0815e640 1 -- 172.21.15.60:0/2366077664 wait complete. 2023-08-24T10:09:23.146 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2023-08-24T10:09:21.545883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:26.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:26.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.269+0000 7f89ab79e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:26.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.273+0000 7f89ab79e640 1 Processor -- start 2023-08-24T10:09:26.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.273+0000 7f89ab79e640 1 -- start start 2023-08-24T10:09:26.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.274+0000 7f89ab79e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f89a4131d40 0x7f89a4134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:26.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.274+0000 7f89ab79e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a4099440 con 0x7f89a406d300 2023-08-24T10:09:26.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.274+0000 7f89ab79e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a409a050 con 0x7f89a4131d40 2023-08-24T10:09:26.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.274+0000 7f89a953a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f89a406d300 0x7f89a412bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38826/0 (socket says 172.21.15.60:38826) 2023-08-24T10:09:26.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.274+0000 7f89a953a640 1 -- 172.21.15.60:0/3291421491 learned_addr learned my addr 172.21.15.60:0/3291421491 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:09:26.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.274+0000 7f89a8d39640 1 --2- 172.21.15.60:0/3291421491 >> v2:172.21.15.60:3300/0 conn(0x7f89a4131d40 0x7f89a4134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:26.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.274+0000 7f899bfff640 1 -- 172.21.15.60:0/3291421491 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 464675121 0 0) 0x7f89a4099440 con 0x7f89a406d300 2023-08-24T10:09:26.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.275+0000 7f899bfff640 1 -- 172.21.15.60:0/3291421491 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89900034f0 con 0x7f89a406d300 2023-08-24T10:09:26.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.275+0000 7f89a8d39640 1 -- 172.21.15.60:0/3291421491 >> v1:172.21.15.60:6789/0 conn(0x7f89a406d300 legacy=0x7f89a412bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:26.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.275+0000 7f89a8d39640 1 -- 172.21.15.60:0/3291421491 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a412b9d0 con 0x7f89a4131d40 2023-08-24T10:09:26.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.275+0000 7f89a8d39640 1 --2- 172.21.15.60:0/3291421491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a4131d40 0x7f89a4134160 crc :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb388b36ed992650 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:26.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.275+0000 7f899bfff640 1 -- 172.21.15.60:0/3291421491 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8994009cd0 con 0x7f89a4131d40 2023-08-24T10:09:26.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.275+0000 7f899bfff640 1 -- 172.21.15.60:0/3291421491 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f8994009e30 con 0x7f89a4131d40 2023-08-24T10:09:26.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.275+0000 7f899bfff640 1 -- 172.21.15.60:0/3291421491 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f899400e360 con 0x7f89a4131d40 2023-08-24T10:09:26.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.276+0000 7f89ab79e640 1 -- 172.21.15.60:0/3291421491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a4131d40 msgr2=0x7f89a4134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:26.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.276+0000 7f89ab79e640 1 --2- 172.21.15.60:0/3291421491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a4131d40 0x7f89a4134160 crc :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:26.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.276+0000 7f89ab79e640 1 -- 172.21.15.60:0/3291421491 shutdown_connections 2023-08-24T10:09:26.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.276+0000 7f89ab79e640 1 --2- 172.21.15.60:0/3291421491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a4131d40 0x7f89a4134160 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:26.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.276+0000 7f89ab79e640 1 -- 172.21.15.60:0/3291421491 >> 172.21.15.60:0/3291421491 conn(0x7f89a409e110 msgr2=0x7f89a409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:26.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.276+0000 7f89ab79e640 1 -- 172.21.15.60:0/3291421491 shutdown_connections 2023-08-24T10:09:26.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.276+0000 7f89ab79e640 1 -- 172.21.15.60:0/3291421491 wait complete. 2023-08-24T10:09:26.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.276+0000 7f89ab79e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:26.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.277+0000 7f89ab79e640 1 Processor -- start 2023-08-24T10:09:26.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.277+0000 7f89ab79e640 1 -- start start 2023-08-24T10:09:26.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.278+0000 7f89ab79e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a406d300 0x7f89a41c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:26.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.278+0000 7f89ab79e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a4127560 con 0x7f89a406d300 2023-08-24T10:09:26.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.278+0000 7f89a953a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a406d300 0x7f89a41c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:26.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.278+0000 7f89a953a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a406d300 0x7f89a41c2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34192/0 (socket says 172.21.15.60:34192) 2023-08-24T10:09:26.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.278+0000 7f89a953a640 1 -- 172.21.15.60:0/1975460968 learned_addr learned my addr 172.21.15.60:0/1975460968 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:26.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.278+0000 7f89a953a640 1 -- 172.21.15.60:0/1975460968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a41290f0 con 0x7f89a406d300 2023-08-24T10:09:26.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.278+0000 7f89a953a640 1 --2- 172.21.15.60:0/1975460968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a406d300 0x7f89a41c2f20 crc :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:26.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.278+0000 7f8999ffb640 1 -- 172.21.15.60:0/1975460968 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f89a00059d0 con 0x7f89a406d300 2023-08-24T10:09:26.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.278+0000 7f8999ffb640 1 -- 172.21.15.60:0/1975460968 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f89a0005b30 con 0x7f89a406d300 2023-08-24T10:09:26.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.278+0000 7f8999ffb640 1 -- 172.21.15.60:0/1975460968 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f89a0005dd0 con 0x7f89a406d300 2023-08-24T10:09:26.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.278+0000 7f89ab79e640 1 -- 172.21.15.60:0/1975460968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a406fe00 con 0x7f89a406d300 2023-08-24T10:09:26.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.279+0000 7f89ab79e640 1 -- 172.21.15.60:0/1975460968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a406eaf0 con 0x7f89a406d300 2023-08-24T10:09:26.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.279+0000 7f8999ffb640 1 -- 172.21.15.60:0/1975460968 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f89a00070b0 con 0x7f89a406d300 2023-08-24T10:09:26.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.280+0000 7f8999ffb640 1 --2- 172.21.15.60:0/1975460968 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8988037130 0x7f8988039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:26.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.280+0000 7f8999ffb640 1 -- 172.21.15.60:0/1975460968 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f89a0046390 con 0x7f89a406d300 2023-08-24T10:09:26.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.280+0000 7f89ab79e640 1 -- 172.21.15.60:0/1975460968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a4061800 con 0x7f89a406d300 2023-08-24T10:09:26.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.280+0000 7f89a8d39640 1 --2- 172.21.15.60:0/1975460968 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8988037130 0x7f8988039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:26.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.280+0000 7f89a8d39640 1 --2- 172.21.15.60:0/1975460968 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8988037130 0x7f8988039640 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:26.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.285+0000 7f8999ffb640 1 -- 172.21.15.60:0/1975460968 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f89a000d720 con 0x7f89a406d300 2023-08-24T10:09:26.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.457+0000 7f89ab79e640 1 -- 172.21.15.60:0/1975460968 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f89a406da20 con 0x7f8988037130 2023-08-24T10:09:26.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.459+0000 7f8999ffb640 1 -- 172.21.15.60:0/1975460968 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f8994009ba0 con 0x7f8988037130 2023-08-24T10:09:26.460 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:26.461 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:26.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.461+0000 7f89ab79e640 1 -- 172.21.15.60:0/1975460968 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8988037130 msgr2=0x7f8988039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:26.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.461+0000 7f89ab79e640 1 --2- 172.21.15.60:0/1975460968 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8988037130 0x7f8988039640 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:26.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.461+0000 7f89ab79e640 1 -- 172.21.15.60:0/1975460968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a406d300 msgr2=0x7f89a41c2f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:26.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.461+0000 7f89ab79e640 1 --2- 172.21.15.60:0/1975460968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a406d300 0x7f89a41c2f20 crc :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:26.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.461+0000 7f89ab79e640 1 -- 172.21.15.60:0/1975460968 shutdown_connections 2023-08-24T10:09:26.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.461+0000 7f89ab79e640 1 --2- 172.21.15.60:0/1975460968 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f8988037130 0x7f8988039640 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:26.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.461+0000 7f89ab79e640 1 --2- 172.21.15.60:0/1975460968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a406d300 0x7f89a41c2f20 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:26.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.461+0000 7f89ab79e640 1 -- 172.21.15.60:0/1975460968 >> 172.21.15.60:0/1975460968 conn(0x7f89a409e110 msgr2=0x7f89a412a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:26.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.461+0000 7f89ab79e640 1 -- 172.21.15.60:0/1975460968 shutdown_connections 2023-08-24T10:09:26.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.461+0000 7f89ab79e640 1 -- 172.21.15.60:0/1975460968 wait complete. 2023-08-24T10:09:26.472 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2023-08-24T10:09:25.546398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:26.473 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:26.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.579+0000 7f346e64e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:26.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.583+0000 7f346e64e640 1 Processor -- start 2023-08-24T10:09:26.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.583+0000 7f346e64e640 1 -- start start 2023-08-24T10:09:26.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.584+0000 7f346e64e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3468071e40 0x7f3468096800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:26.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.584+0000 7f346e64e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34680977f0 con 0x7f346806f100 2023-08-24T10:09:26.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.584+0000 7f346e64e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f346809a280 con 0x7f3468071e40 2023-08-24T10:09:26.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.584+0000 7f3467fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3468071e40 0x7f3468096800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:26.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.584+0000 7f3467fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3468071e40 0x7f3468096800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51714/0 (socket says 172.21.15.60:51714) 2023-08-24T10:09:26.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.584+0000 7f3467fff640 1 -- 172.21.15.60:0/503569845 learned_addr learned my addr 172.21.15.60:0/503569845 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:26.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.584+0000 7f3467fff640 1 -- 172.21.15.60:0/503569845 >> v1:172.21.15.60:6789/0 conn(0x7f346806f100 legacy=0x7f346806f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:26.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.584+0000 7f3467fff640 1 -- 172.21.15.60:0/503569845 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346809a6b0 con 0x7f3468071e40 2023-08-24T10:09:26.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.585+0000 7f3467fff640 1 --2- 172.21.15.60:0/503569845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3468071e40 0x7f3468096800 crc :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdff98f037291db2 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:26.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.585+0000 7f3466ffd640 1 -- 172.21.15.60:0/503569845 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3450009dd0 con 0x7f3468071e40 2023-08-24T10:09:26.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.585+0000 7f3466ffd640 1 -- 172.21.15.60:0/503569845 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f345000e230 con 0x7f3468071e40 2023-08-24T10:09:26.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.585+0000 7f3466ffd640 1 -- 172.21.15.60:0/503569845 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f345000e4d0 con 0x7f3468071e40 2023-08-24T10:09:26.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.585+0000 7f346e64e640 1 -- 172.21.15.60:0/503569845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3468071e40 msgr2=0x7f3468096800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:26.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.585+0000 7f346e64e640 1 --2- 172.21.15.60:0/503569845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3468071e40 0x7f3468096800 crc :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:26.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.585+0000 7f346e64e640 1 -- 172.21.15.60:0/503569845 shutdown_connections 2023-08-24T10:09:26.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.585+0000 7f346e64e640 1 --2- 172.21.15.60:0/503569845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3468071e40 0x7f3468096800 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:26.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.585+0000 7f346e64e640 1 -- 172.21.15.60:0/503569845 >> 172.21.15.60:0/503569845 conn(0x7f346809e110 msgr2=0x7f346809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:26.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.585+0000 7f346e64e640 1 -- 172.21.15.60:0/503569845 shutdown_connections 2023-08-24T10:09:26.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.585+0000 7f346e64e640 1 -- 172.21.15.60:0/503569845 wait complete. 2023-08-24T10:09:26.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.586+0000 7f346e64e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:26.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.587+0000 7f346e64e640 1 Processor -- start 2023-08-24T10:09:26.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.587+0000 7f346e64e640 1 -- start start 2023-08-24T10:09:26.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.587+0000 7f346e64e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f346806f100 0x7f34681c19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:26.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.587+0000 7f346e64e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346808ffe0 con 0x7f346806f100 2023-08-24T10:09:26.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.587+0000 7f3467fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f346806f100 0x7f34681c19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:26.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.587+0000 7f3467fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f346806f100 0x7f34681c19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51728/0 (socket says 172.21.15.60:51728) 2023-08-24T10:09:26.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.587+0000 7f3467fff640 1 -- 172.21.15.60:0/2627065324 learned_addr learned my addr 172.21.15.60:0/2627065324 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:26.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.588+0000 7f3467fff640 1 -- 172.21.15.60:0/2627065324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34681c14b0 con 0x7f346806f100 2023-08-24T10:09:26.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.588+0000 7f3467fff640 1 --2- 172.21.15.60:0/2627065324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f346806f100 0x7f34681c19f0 crc :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:26.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.588+0000 7f3464ff9640 1 -- 172.21.15.60:0/2627065324 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3450014540 con 0x7f346806f100 2023-08-24T10:09:26.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.588+0000 7f3464ff9640 1 -- 172.21.15.60:0/2627065324 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f34500146a0 con 0x7f346806f100 2023-08-24T10:09:26.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.588+0000 7f3464ff9640 1 -- 172.21.15.60:0/2627065324 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3450014940 con 0x7f346806f100 2023-08-24T10:09:26.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.588+0000 7f346e64e640 1 -- 172.21.15.60:0/2627065324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3468090720 con 0x7f346806f100 2023-08-24T10:09:26.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.588+0000 7f346e64e640 1 -- 172.21.15.60:0/2627065324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3468092c10 con 0x7f346806f100 2023-08-24T10:09:26.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.589+0000 7f3464ff9640 1 -- 172.21.15.60:0/2627065324 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f3450014af0 con 0x7f346806f100 2023-08-24T10:09:26.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.589+0000 7f346e64e640 1 -- 172.21.15.60:0/2627065324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3468061800 con 0x7f346806f100 2023-08-24T10:09:26.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.589+0000 7f3464ff9640 1 --2- 172.21.15.60:0/2627065324 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3448037180 0x7f3448039690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:26.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.589+0000 7f3464ff9640 1 -- 172.21.15.60:0/2627065324 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f3450019040 con 0x7f346806f100 2023-08-24T10:09:26.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.590+0000 7f34677fe640 1 --2- 172.21.15.60:0/2627065324 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3448037180 0x7f3448039690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:26.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.590+0000 7f34677fe640 1 --2- 172.21.15.60:0/2627065324 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3448037180 0x7f3448039690 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:26.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.594+0000 7f3464ff9640 1 -- 172.21.15.60:0/2627065324 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f3450018190 con 0x7f346806f100 2023-08-24T10:09:26.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.766+0000 7f346e64e640 1 -- 172.21.15.60:0/2627065324 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3468097a80 con 0x7f3448037180 2023-08-24T10:09:26.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.767+0000 7f3464ff9640 1 -- 172.21.15.60:0/2627065324 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f3458001650 con 0x7f3448037180 2023-08-24T10:09:26.769 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:26.770 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:26.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.769+0000 7f346e64e640 1 -- 172.21.15.60:0/2627065324 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3448037180 msgr2=0x7f3448039690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:26.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.769+0000 7f346e64e640 1 --2- 172.21.15.60:0/2627065324 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3448037180 0x7f3448039690 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:26.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.769+0000 7f346e64e640 1 -- 172.21.15.60:0/2627065324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f346806f100 msgr2=0x7f34681c19f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:26.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.769+0000 7f346e64e640 1 --2- 172.21.15.60:0/2627065324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f346806f100 0x7f34681c19f0 crc :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:26.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.769+0000 7f346e64e640 1 -- 172.21.15.60:0/2627065324 shutdown_connections 2023-08-24T10:09:26.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.769+0000 7f346e64e640 1 --2- 172.21.15.60:0/2627065324 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3448037180 0x7f3448039690 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:26.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.769+0000 7f346e64e640 1 --2- 172.21.15.60:0/2627065324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f346806f100 0x7f34681c19f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:26.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.769+0000 7f346e64e640 1 -- 172.21.15.60:0/2627065324 >> 172.21.15.60:0/2627065324 conn(0x7f346809e110 msgr2=0x7f34680985b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:26.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.769+0000 7f346e64e640 1 -- 172.21.15.60:0/2627065324 shutdown_connections 2023-08-24T10:09:26.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.769+0000 7f346e64e640 1 -- 172.21.15.60:0/2627065324 wait complete. 2023-08-24T10:09:26.780 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2023-08-24T10:09:25.546398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:26.781 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:09:26.782 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:26.782 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:09:26.782 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:26.782 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:09:26.783 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:26.783 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:09:26.783 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:26.784 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:09:26.784 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:26.785 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:09:26.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:26.785 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:09:26.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:26.786 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:09:26.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:26.787 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:09:26.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:26.787 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:09:26.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.894+0000 7ffba2e30640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:26.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.897+0000 7ffba2e30640 1 Processor -- start 2023-08-24T10:09:26.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.897+0000 7ffba2e30640 1 -- start start 2023-08-24T10:09:26.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.898+0000 7ffba2e30640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffb9c132680 0x7ffb9c134aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:26.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.898+0000 7ffba2e30640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb9c06dfc0 con 0x7ffb9c070c30 2023-08-24T10:09:26.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.898+0000 7ffba2e30640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb9c06d9b0 con 0x7ffb9c132680 2023-08-24T10:09:26.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.898+0000 7ffb9bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffb9c132680 0x7ffb9c134aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:26.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.898+0000 7ffb9bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffb9c132680 0x7ffb9c134aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51730/0 (socket says 172.21.15.60:51730) 2023-08-24T10:09:26.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.898+0000 7ffb9bfff640 1 -- 172.21.15.60:0/704105053 learned_addr learned my addr 172.21.15.60:0/704105053 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:26.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.898+0000 7ffba0bcc640 1 --1- 172.21.15.60:0/704105053 >> v1:172.21.15.60:6789/0 conn(0x7ffb9c070c30 0x7ffb9c071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38236/0 (socket says 172.21.15.60:38236) 2023-08-24T10:09:26.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.898+0000 7ffb9bfff640 1 -- 172.21.15.60:0/704105053 >> v1:172.21.15.60:6789/0 conn(0x7ffb9c070c30 legacy=0x7ffb9c071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:26.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.898+0000 7ffb9bfff640 1 -- 172.21.15.60:0/704105053 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c06e6e0 con 0x7ffb9c132680 2023-08-24T10:09:26.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.898+0000 7ffb9bfff640 1 --2- 172.21.15.60:0/704105053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb9c132680 0x7ffb9c134aa0 crc :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de9a0b1b2c4f062 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:26.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.898+0000 7ffb9b7fe640 1 -- 172.21.15.60:0/704105053 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ffb88009dd0 con 0x7ffb9c132680 2023-08-24T10:09:26.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.898+0000 7ffb9b7fe640 1 -- 172.21.15.60:0/704105053 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ffb8800e230 con 0x7ffb9c132680 2023-08-24T10:09:26.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.898+0000 7ffb9b7fe640 1 -- 172.21.15.60:0/704105053 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ffb8800e4d0 con 0x7ffb9c132680 2023-08-24T10:09:26.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.899+0000 7ffba2e30640 1 -- 172.21.15.60:0/704105053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb9c132680 msgr2=0x7ffb9c134aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:26.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.899+0000 7ffba2e30640 1 --2- 172.21.15.60:0/704105053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb9c132680 0x7ffb9c134aa0 crc :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:26.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.899+0000 7ffba2e30640 1 -- 172.21.15.60:0/704105053 shutdown_connections 2023-08-24T10:09:26.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.899+0000 7ffba2e30640 1 --2- 172.21.15.60:0/704105053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb9c132680 0x7ffb9c134aa0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:26.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.899+0000 7ffba2e30640 1 -- 172.21.15.60:0/704105053 >> 172.21.15.60:0/704105053 conn(0x7ffb9c09e110 msgr2=0x7ffb9c09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:26.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.899+0000 7ffba2e30640 1 -- 172.21.15.60:0/704105053 shutdown_connections 2023-08-24T10:09:26.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.899+0000 7ffba2e30640 1 -- 172.21.15.60:0/704105053 wait complete. 2023-08-24T10:09:26.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.899+0000 7ffba2e30640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:26.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.900+0000 7ffba2e30640 1 Processor -- start 2023-08-24T10:09:26.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.900+0000 7ffba2e30640 1 -- start start 2023-08-24T10:09:26.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.901+0000 7ffba2e30640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb9c070c30 0x7ffb9c12c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:26.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.901+0000 7ffba2e30640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c12f4a0 con 0x7ffb9c070c30 2023-08-24T10:09:26.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.901+0000 7ffba0bcc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb9c070c30 0x7ffb9c12c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:26.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.901+0000 7ffba0bcc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb9c070c30 0x7ffb9c12c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51746/0 (socket says 172.21.15.60:51746) 2023-08-24T10:09:26.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.901+0000 7ffba0bcc640 1 -- 172.21.15.60:0/1927546998 learned_addr learned my addr 172.21.15.60:0/1927546998 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:26.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.901+0000 7ffba0bcc640 1 -- 172.21.15.60:0/1927546998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c1300b0 con 0x7ffb9c070c30 2023-08-24T10:09:26.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.902+0000 7ffba0bcc640 1 --2- 172.21.15.60:0/1927546998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb9c070c30 0x7ffb9c12c1a0 crc :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:26.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.902+0000 7ffb997fa640 1 -- 172.21.15.60:0/1927546998 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ffb900071e0 con 0x7ffb9c070c30 2023-08-24T10:09:26.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.902+0000 7ffb997fa640 1 -- 172.21.15.60:0/1927546998 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ffb90007340 con 0x7ffb9c070c30 2023-08-24T10:09:26.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.902+0000 7ffba2e30640 1 -- 172.21.15.60:0/1927546998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9c098b50 con 0x7ffb9c070c30 2023-08-24T10:09:26.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.902+0000 7ffb997fa640 1 -- 172.21.15.60:0/1927546998 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ffb900075e0 con 0x7ffb9c070c30 2023-08-24T10:09:26.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.902+0000 7ffba2e30640 1 -- 172.21.15.60:0/1927546998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9c12dd40 con 0x7ffb9c070c30 2023-08-24T10:09:26.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.903+0000 7ffb997fa640 1 -- 172.21.15.60:0/1927546998 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7ffb900057e0 con 0x7ffb9c070c30 2023-08-24T10:09:26.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.903+0000 7ffba2e30640 1 -- 172.21.15.60:0/1927546998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb9c061800 con 0x7ffb9c070c30 2023-08-24T10:09:26.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.903+0000 7ffb997fa640 1 --2- 172.21.15.60:0/1927546998 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ffb80037130 0x7ffb80039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:26.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.903+0000 7ffb997fa640 1 -- 172.21.15.60:0/1927546998 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7ffb90046340 con 0x7ffb9c070c30 2023-08-24T10:09:26.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.903+0000 7ffb9bfff640 1 --2- 172.21.15.60:0/1927546998 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ffb80037130 0x7ffb80039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:26.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.904+0000 7ffb9bfff640 1 --2- 172.21.15.60:0/1927546998 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ffb80037130 0x7ffb80039640 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:26.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:26.907+0000 7ffb997fa640 1 -- 172.21.15.60:0/1927546998 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7ffb9000d330 con 0x7ffb9c070c30 2023-08-24T10:09:27.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.151+0000 7ffba2e30640 1 -- 172.21.15.60:0/1927546998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ffb9c0706e0 con 0x7ffb9c070c30 2023-08-24T10:09:27.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.151+0000 7ffb997fa640 1 -- 172.21.15.60:0/1927546998 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7ffb9000d150 con 0x7ffb9c070c30 2023-08-24T10:09:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:27.153 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":277,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:09:27.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.153+0000 7ffba2e30640 1 -- 172.21.15.60:0/1927546998 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ffb80037130 msgr2=0x7ffb80039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:27.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.153+0000 7ffba2e30640 1 --2- 172.21.15.60:0/1927546998 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ffb80037130 0x7ffb80039640 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:27.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.153+0000 7ffba2e30640 1 -- 172.21.15.60:0/1927546998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb9c070c30 msgr2=0x7ffb9c12c1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:27.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.153+0000 7ffba2e30640 1 --2- 172.21.15.60:0/1927546998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb9c070c30 0x7ffb9c12c1a0 crc :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:27.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.154+0000 7ffba2e30640 1 -- 172.21.15.60:0/1927546998 shutdown_connections 2023-08-24T10:09:27.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.154+0000 7ffba2e30640 1 --2- 172.21.15.60:0/1927546998 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ffb80037130 0x7ffb80039640 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:27.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.154+0000 7ffba2e30640 1 --2- 172.21.15.60:0/1927546998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb9c070c30 0x7ffb9c12c1a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:27.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.154+0000 7ffba2e30640 1 -- 172.21.15.60:0/1927546998 >> 172.21.15.60:0/1927546998 conn(0x7ffb9c09e110 msgr2=0x7ffb9c1b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:27.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.154+0000 7ffba2e30640 1 -- 172.21.15.60:0/1927546998 shutdown_connections 2023-08-24T10:09:27.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.154+0000 7ffba2e30640 1 -- 172.21.15.60:0/1927546998 wait complete. 2023-08-24T10:09:27.165 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:09:27.165 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:27.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.272+0000 7f1cea71e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:27.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.276+0000 7f1cea71e640 1 Processor -- start 2023-08-24T10:09:27.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.276+0000 7f1cea71e640 1 -- start start 2023-08-24T10:09:27.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.276+0000 7f1cea71e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1ce4096860 0x7f1ce4096c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:27.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.276+0000 7f1cea71e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce40971b0 con 0x7f1ce40990b0 2023-08-24T10:09:27.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.276+0000 7f1cea71e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce406db70 con 0x7f1ce4096860 2023-08-24T10:09:27.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.277+0000 7f1ce3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1ce4096860 0x7f1ce4096c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:27.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.277+0000 7f1ce3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1ce4096860 0x7f1ce4096c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51748/0 (socket says 172.21.15.60:51748) 2023-08-24T10:09:27.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.277+0000 7f1ce3fff640 1 -- 172.21.15.60:0/2935241982 learned_addr learned my addr 172.21.15.60:0/2935241982 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:27.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.277+0000 7f1ce37fe640 1 --1- 172.21.15.60:0/2935241982 >> v1:172.21.15.60:6789/0 conn(0x7f1ce40990b0 0x7f1ce40994c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38240/0 (socket says 172.21.15.60:38240) 2023-08-24T10:09:27.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.277+0000 7f1ce3fff640 1 -- 172.21.15.60:0/2935241982 >> v1:172.21.15.60:6789/0 conn(0x7f1ce40990b0 legacy=0x7f1ce40994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:27.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.277+0000 7f1ce3fff640 1 -- 172.21.15.60:0/2935241982 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce406e320 con 0x7f1ce4096860 2023-08-24T10:09:27.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.277+0000 7f1ce3fff640 1 --2- 172.21.15.60:0/2935241982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ce4096860 0x7f1ce4096c70 crc :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83519902b6e4d477 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:27.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.277+0000 7f1ce2ffd640 1 -- 172.21.15.60:0/2935241982 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1cd4009dd0 con 0x7f1ce4096860 2023-08-24T10:09:27.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.277+0000 7f1ce2ffd640 1 -- 172.21.15.60:0/2935241982 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1cd400e230 con 0x7f1ce4096860 2023-08-24T10:09:27.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.277+0000 7f1ce2ffd640 1 -- 172.21.15.60:0/2935241982 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1cd400e4d0 con 0x7f1ce4096860 2023-08-24T10:09:27.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.278+0000 7f1cea71e640 1 -- 172.21.15.60:0/2935241982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ce4096860 msgr2=0x7f1ce4096c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:27.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.278+0000 7f1cea71e640 1 --2- 172.21.15.60:0/2935241982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ce4096860 0x7f1ce4096c70 crc :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:27.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.278+0000 7f1cea71e640 1 -- 172.21.15.60:0/2935241982 shutdown_connections 2023-08-24T10:09:27.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.278+0000 7f1cea71e640 1 --2- 172.21.15.60:0/2935241982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ce4096860 0x7f1ce4096c70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:27.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.278+0000 7f1cea71e640 1 -- 172.21.15.60:0/2935241982 >> 172.21.15.60:0/2935241982 conn(0x7f1ce409e110 msgr2=0x7f1ce409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:27.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.278+0000 7f1cea71e640 1 -- 172.21.15.60:0/2935241982 shutdown_connections 2023-08-24T10:09:27.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.278+0000 7f1cea71e640 1 -- 172.21.15.60:0/2935241982 wait complete. 2023-08-24T10:09:27.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.278+0000 7f1cea71e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:27.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.279+0000 7f1cea71e640 1 Processor -- start 2023-08-24T10:09:27.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.279+0000 7f1cea71e640 1 -- start start 2023-08-24T10:09:27.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.280+0000 7f1cea71e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ce4096860 0x7f1ce4126ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:27.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.280+0000 7f1cea71e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce4130220 con 0x7f1ce4096860 2023-08-24T10:09:27.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.280+0000 7f1ce3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ce4096860 0x7f1ce4126ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:27.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.280+0000 7f1ce3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ce4096860 0x7f1ce4126ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51764/0 (socket says 172.21.15.60:51764) 2023-08-24T10:09:27.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.280+0000 7f1ce3fff640 1 -- 172.21.15.60:0/2615290148 learned_addr learned my addr 172.21.15.60:0/2615290148 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:27.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.280+0000 7f1ce3fff640 1 -- 172.21.15.60:0/2615290148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce4128a90 con 0x7f1ce4096860 2023-08-24T10:09:27.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.280+0000 7f1ce3fff640 1 --2- 172.21.15.60:0/2615290148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ce4096860 0x7f1ce4126ef0 crc :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:27.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.280+0000 7f1ce0ff9640 1 -- 172.21.15.60:0/2615290148 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1cd4014540 con 0x7f1ce4096860 2023-08-24T10:09:27.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.280+0000 7f1ce0ff9640 1 -- 172.21.15.60:0/2615290148 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1cd40146a0 con 0x7f1ce4096860 2023-08-24T10:09:27.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.280+0000 7f1ce0ff9640 1 -- 172.21.15.60:0/2615290148 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1cd4014940 con 0x7f1ce4096860 2023-08-24T10:09:27.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.280+0000 7f1cea71e640 1 -- 172.21.15.60:0/2615290148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce412ff10 con 0x7f1ce4096860 2023-08-24T10:09:27.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.281+0000 7f1cea71e640 1 -- 172.21.15.60:0/2615290148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce412a370 con 0x7f1ce4096860 2023-08-24T10:09:27.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.282+0000 7f1ce0ff9640 1 -- 172.21.15.60:0/2615290148 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f1cd4014af0 con 0x7f1ce4096860 2023-08-24T10:09:27.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.282+0000 7f1ce0ff9640 1 --2- 172.21.15.60:0/2615290148 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1cd0037130 0x7f1cd0039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:27.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.282+0000 7f1cea71e640 1 -- 172.21.15.60:0/2615290148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce412ade0 con 0x7f1ce4096860 2023-08-24T10:09:27.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.282+0000 7f1ce0ff9640 1 -- 172.21.15.60:0/2615290148 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f1cd4019040 con 0x7f1ce4096860 2023-08-24T10:09:27.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.282+0000 7f1ce37fe640 1 --2- 172.21.15.60:0/2615290148 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1cd0037130 0x7f1cd0039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:27.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.282+0000 7f1ce37fe640 1 --2- 172.21.15.60:0/2615290148 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1cd0037130 0x7f1cd0039640 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:27.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.286+0000 7f1ce0ff9640 1 -- 172.21.15.60:0/2615290148 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f1ce412ade0 con 0x7f1ce4096860 2023-08-24T10:09:27.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.459+0000 7f1cea71e640 1 -- 172.21.15.60:0/2615290148 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ce406d190 con 0x7f1cd0037130 2023-08-24T10:09:27.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.460+0000 7f1ce0ff9640 1 -- 172.21.15.60:0/2615290148 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f1cd80070b0 con 0x7f1cd0037130 2023-08-24T10:09:27.461 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:27.462 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:27.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.462+0000 7f1cea71e640 1 -- 172.21.15.60:0/2615290148 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1cd0037130 msgr2=0x7f1cd0039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:27.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.462+0000 7f1cea71e640 1 --2- 172.21.15.60:0/2615290148 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1cd0037130 0x7f1cd0039640 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:27.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.462+0000 7f1cea71e640 1 -- 172.21.15.60:0/2615290148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ce4096860 msgr2=0x7f1ce4126ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:27.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.462+0000 7f1cea71e640 1 --2- 172.21.15.60:0/2615290148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ce4096860 0x7f1ce4126ef0 crc :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:27.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.462+0000 7f1cea71e640 1 -- 172.21.15.60:0/2615290148 shutdown_connections 2023-08-24T10:09:27.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.462+0000 7f1cea71e640 1 --2- 172.21.15.60:0/2615290148 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1cd0037130 0x7f1cd0039640 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:27.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.462+0000 7f1cea71e640 1 --2- 172.21.15.60:0/2615290148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ce4096860 0x7f1ce4126ef0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:27.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.462+0000 7f1cea71e640 1 -- 172.21.15.60:0/2615290148 >> 172.21.15.60:0/2615290148 conn(0x7f1ce409e110 msgr2=0x7f1ce41b73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:27.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.462+0000 7f1cea71e640 1 -- 172.21.15.60:0/2615290148 shutdown_connections 2023-08-24T10:09:27.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:27.462+0000 7f1cea71e640 1 -- 172.21.15.60:0/2615290148 wait complete. 2023-08-24T10:09:27.473 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2023-08-24T10:09:25.546398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:27.645 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~260s 2023-08-24T10:09:27.645 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~260s 2023-08-24T10:09:27.646 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~260s 2023-08-24T10:09:27.646 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~260s 2023-08-24T10:09:30.475 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:30.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.596+0000 7f2bd8eec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:30.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.600+0000 7f2bd8eec640 1 Processor -- start 2023-08-24T10:09:30.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.600+0000 7f2bd8eec640 1 -- start start 2023-08-24T10:09:30.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.601+0000 7f2bd8eec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2bd4131d40 0x7f2bd4134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:30.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.601+0000 7f2bd8eec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd4099440 con 0x7f2bd406d300 2023-08-24T10:09:30.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.601+0000 7f2bd8eec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd409a050 con 0x7f2bd4131d40 2023-08-24T10:09:30.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.601+0000 7f2bcbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2bd4131d40 0x7f2bd4134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:30.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.601+0000 7f2bcbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2bd4131d40 0x7f2bd4134160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51778/0 (socket says 172.21.15.60:51778) 2023-08-24T10:09:30.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.601+0000 7f2bcbfff640 1 -- 172.21.15.60:0/293620872 learned_addr learned my addr 172.21.15.60:0/293620872 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:30.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.601+0000 7f2bcbfff640 1 -- 172.21.15.60:0/293620872 >> v1:172.21.15.60:6789/0 conn(0x7f2bd406d300 legacy=0x7f2bd412bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:30.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.601+0000 7f2bcbfff640 1 -- 172.21.15.60:0/293620872 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd412b9d0 con 0x7f2bd4131d40 2023-08-24T10:09:30.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.601+0000 7f2bcbfff640 1 --2- 172.21.15.60:0/293620872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bd4131d40 0x7f2bd4134160 crc :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7547afa395c8527 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:30.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.602+0000 7f2bd1d9b640 1 -- 172.21.15.60:0/293620872 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2bbc00ad50 con 0x7f2bd4131d40 2023-08-24T10:09:30.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.602+0000 7f2bd1d9b640 1 -- 172.21.15.60:0/293620872 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f2bbc00f340 con 0x7f2bd4131d40 2023-08-24T10:09:30.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.602+0000 7f2bd1d9b640 1 -- 172.21.15.60:0/293620872 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2bbc00f5e0 con 0x7f2bd4131d40 2023-08-24T10:09:30.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.602+0000 7f2bd8eec640 1 -- 172.21.15.60:0/293620872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bd4131d40 msgr2=0x7f2bd4134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:30.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.602+0000 7f2bd8eec640 1 --2- 172.21.15.60:0/293620872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bd4131d40 0x7f2bd4134160 crc :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:30.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.602+0000 7f2bd8eec640 1 -- 172.21.15.60:0/293620872 shutdown_connections 2023-08-24T10:09:30.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.602+0000 7f2bd8eec640 1 --2- 172.21.15.60:0/293620872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bd4131d40 0x7f2bd4134160 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:30.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.602+0000 7f2bd8eec640 1 -- 172.21.15.60:0/293620872 >> 172.21.15.60:0/293620872 conn(0x7f2bd409e110 msgr2=0x7f2bd409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:30.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.602+0000 7f2bd8eec640 1 -- 172.21.15.60:0/293620872 shutdown_connections 2023-08-24T10:09:30.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.602+0000 7f2bd8eec640 1 -- 172.21.15.60:0/293620872 wait complete. 2023-08-24T10:09:30.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.603+0000 7f2bd8eec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:30.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.604+0000 7f2bd8eec640 1 Processor -- start 2023-08-24T10:09:30.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.604+0000 7f2bd8eec640 1 -- start start 2023-08-24T10:09:30.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.604+0000 7f2bd8eec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bd406d300 0x7f2bd41c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:30.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.604+0000 7f2bd8eec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd4127580 con 0x7f2bd406d300 2023-08-24T10:09:30.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.604+0000 7f2bd259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bd406d300 0x7f2bd41c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:30.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.604+0000 7f2bd259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bd406d300 0x7f2bd41c2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51782/0 (socket says 172.21.15.60:51782) 2023-08-24T10:09:30.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.605+0000 7f2bd259c640 1 -- 172.21.15.60:0/3796487856 learned_addr learned my addr 172.21.15.60:0/3796487856 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:30.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.605+0000 7f2bd259c640 1 -- 172.21.15.60:0/3796487856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd41290d0 con 0x7f2bd406d300 2023-08-24T10:09:30.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.605+0000 7f2bd259c640 1 --2- 172.21.15.60:0/3796487856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bd406d300 0x7f2bd41c2f20 crc :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:30.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.605+0000 7f2bcaffd640 1 -- 172.21.15.60:0/3796487856 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2bc400eb70 con 0x7f2bd406d300 2023-08-24T10:09:30.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.605+0000 7f2bcaffd640 1 -- 172.21.15.60:0/3796487856 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f2bc400ecd0 con 0x7f2bd406d300 2023-08-24T10:09:30.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.605+0000 7f2bd8eec640 1 -- 172.21.15.60:0/3796487856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd40718b0 con 0x7f2bd406d300 2023-08-24T10:09:30.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.605+0000 7f2bcaffd640 1 -- 172.21.15.60:0/3796487856 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2bc4011440 con 0x7f2bd406d300 2023-08-24T10:09:30.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.605+0000 7f2bd8eec640 1 -- 172.21.15.60:0/3796487856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd406f9f0 con 0x7f2bd406d300 2023-08-24T10:09:30.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.606+0000 7f2bcaffd640 1 -- 172.21.15.60:0/3796487856 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f2bc40115f0 con 0x7f2bd406d300 2023-08-24T10:09:30.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.606+0000 7f2bd8eec640 1 -- 172.21.15.60:0/3796487856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd4061800 con 0x7f2bd406d300 2023-08-24T10:09:30.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.606+0000 7f2bcaffd640 1 --2- 172.21.15.60:0/3796487856 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2bb4037180 0x7f2bb4039690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:30.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.606+0000 7f2bcaffd640 1 -- 172.21.15.60:0/3796487856 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f2bc4049100 con 0x7f2bd406d300 2023-08-24T10:09:30.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.607+0000 7f2bcbfff640 1 --2- 172.21.15.60:0/3796487856 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2bb4037180 0x7f2bb4039690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:30.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.607+0000 7f2bcbfff640 1 --2- 172.21.15.60:0/3796487856 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2bb4037180 0x7f2bb4039690 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:30.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.611+0000 7f2bcaffd640 1 -- 172.21.15.60:0/3796487856 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f2bc4014510 con 0x7f2bd406d300 2023-08-24T10:09:30.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.784+0000 7f2bd8eec640 1 -- 172.21.15.60:0/3796487856 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2bd412a310 con 0x7f2bb4037180 2023-08-24T10:09:30.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.785+0000 7f2bcaffd640 1 -- 172.21.15.60:0/3796487856 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f2bbc00ad20 con 0x7f2bb4037180 2023-08-24T10:09:30.786 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:30.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.787+0000 7f2bd8eec640 1 -- 172.21.15.60:0/3796487856 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2bb4037180 msgr2=0x7f2bb4039690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:30.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.787+0000 7f2bd8eec640 1 --2- 172.21.15.60:0/3796487856 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2bb4037180 0x7f2bb4039690 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:30.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.787+0000 7f2bd8eec640 1 -- 172.21.15.60:0/3796487856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bd406d300 msgr2=0x7f2bd41c2f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:30.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.787+0000 7f2bd8eec640 1 --2- 172.21.15.60:0/3796487856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bd406d300 0x7f2bd41c2f20 crc :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:30.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.787+0000 7f2bd8eec640 1 -- 172.21.15.60:0/3796487856 shutdown_connections 2023-08-24T10:09:30.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.787+0000 7f2bd8eec640 1 --2- 172.21.15.60:0/3796487856 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2bb4037180 0x7f2bb4039690 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:30.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.787+0000 7f2bd8eec640 1 --2- 172.21.15.60:0/3796487856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bd406d300 0x7f2bd41c2f20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:30.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.787+0000 7f2bd8eec640 1 -- 172.21.15.60:0/3796487856 >> 172.21.15.60:0/3796487856 conn(0x7f2bd409e110 msgr2=0x7f2bd412ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:30.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.787+0000 7f2bd8eec640 1 -- 172.21.15.60:0/3796487856 shutdown_connections 2023-08-24T10:09:30.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.787+0000 7f2bd8eec640 1 -- 172.21.15.60:0/3796487856 wait complete. 2023-08-24T10:09:30.798 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2023-08-24T10:09:29.547277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:30.800 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:30.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.907+0000 7fc72eaac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:30.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.910+0000 7fc72eaac640 1 Processor -- start 2023-08-24T10:09:30.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.911+0000 7fc72eaac640 1 -- start start 2023-08-24T10:09:30.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.911+0000 7fc72eaac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc728131d40 0x7fc728134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:30.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.911+0000 7fc72eaac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc728099440 con 0x7fc72806d300 2023-08-24T10:09:30.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.911+0000 7fc72eaac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc72809a050 con 0x7fc728131d40 2023-08-24T10:09:30.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.911+0000 7fc72c848640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc72806d300 0x7fc72812bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38258/0 (socket says 172.21.15.60:38258) 2023-08-24T10:09:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.911+0000 7fc72c848640 1 -- 172.21.15.60:0/844508644 learned_addr learned my addr 172.21.15.60:0/844508644 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:09:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.911+0000 7fc727fff640 1 --2- 172.21.15.60:0/844508644 >> v2:172.21.15.60:3300/0 conn(0x7fc728131d40 0x7fc728134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.912+0000 7fc7277fe640 1 -- 172.21.15.60:0/844508644 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 673087857 0 0) 0x7fc728099440 con 0x7fc72806d300 2023-08-24T10:09:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.912+0000 7fc7277fe640 1 -- 172.21.15.60:0/844508644 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc710003410 con 0x7fc72806d300 2023-08-24T10:09:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.912+0000 7fc727fff640 1 -- 172.21.15.60:0/844508644 >> v1:172.21.15.60:6789/0 conn(0x7fc72806d300 legacy=0x7fc72812bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.912+0000 7fc727fff640 1 -- 172.21.15.60:0/844508644 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72812b9d0 con 0x7fc728131d40 2023-08-24T10:09:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.912+0000 7fc727fff640 1 --2- 172.21.15.60:0/844508644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc728131d40 0x7fc728134160 crc :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22978e188d86435d server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.912+0000 7fc7277fe640 1 -- 172.21.15.60:0/844508644 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc71c009cd0 con 0x7fc728131d40 2023-08-24T10:09:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.912+0000 7fc7277fe640 1 -- 172.21.15.60:0/844508644 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc71c009e30 con 0x7fc728131d40 2023-08-24T10:09:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.912+0000 7fc7277fe640 1 -- 172.21.15.60:0/844508644 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc71c00e360 con 0x7fc728131d40 2023-08-24T10:09:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.913+0000 7fc72eaac640 1 -- 172.21.15.60:0/844508644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc728131d40 msgr2=0x7fc728134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.913+0000 7fc72eaac640 1 --2- 172.21.15.60:0/844508644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc728131d40 0x7fc728134160 crc :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.913+0000 7fc72eaac640 1 -- 172.21.15.60:0/844508644 shutdown_connections 2023-08-24T10:09:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.913+0000 7fc72eaac640 1 --2- 172.21.15.60:0/844508644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc728131d40 0x7fc728134160 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.913+0000 7fc72eaac640 1 -- 172.21.15.60:0/844508644 >> 172.21.15.60:0/844508644 conn(0x7fc72809e110 msgr2=0x7fc72809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.913+0000 7fc72eaac640 1 -- 172.21.15.60:0/844508644 shutdown_connections 2023-08-24T10:09:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.913+0000 7fc72eaac640 1 -- 172.21.15.60:0/844508644 wait complete. 2023-08-24T10:09:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.913+0000 7fc72eaac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.914+0000 7fc72eaac640 1 Processor -- start 2023-08-24T10:09:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.914+0000 7fc72eaac640 1 -- start start 2023-08-24T10:09:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.915+0000 7fc72eaac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc72806d300 0x7fc728126450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.915+0000 7fc72eaac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72806ee10 con 0x7fc72806d300 2023-08-24T10:09:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.915+0000 7fc72c848640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc72806d300 0x7fc728126450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.915+0000 7fc72c848640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc72806d300 0x7fc728126450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51810/0 (socket says 172.21.15.60:51810) 2023-08-24T10:09:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.915+0000 7fc72c848640 1 -- 172.21.15.60:0/3248023249 learned_addr learned my addr 172.21.15.60:0/3248023249 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.915+0000 7fc72c848640 1 -- 172.21.15.60:0/3248023249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72806df20 con 0x7fc72806d300 2023-08-24T10:09:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.915+0000 7fc72c848640 1 --2- 172.21.15.60:0/3248023249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc72806d300 0x7fc728126450 crc :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.915+0000 7fc7257fa640 1 -- 172.21.15.60:0/3248023249 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc7200070b0 con 0x7fc72806d300 2023-08-24T10:09:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.915+0000 7fc7257fa640 1 -- 172.21.15.60:0/3248023249 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc720007210 con 0x7fc72806d300 2023-08-24T10:09:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.915+0000 7fc72eaac640 1 -- 172.21.15.60:0/3248023249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72806e1e0 con 0x7fc72806d300 2023-08-24T10:09:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.915+0000 7fc7257fa640 1 -- 172.21.15.60:0/3248023249 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc7200074b0 con 0x7fc72806d300 2023-08-24T10:09:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.915+0000 7fc72eaac640 1 -- 172.21.15.60:0/3248023249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc728071be0 con 0x7fc72806d300 2023-08-24T10:09:30.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.916+0000 7fc7257fa640 1 -- 172.21.15.60:0/3248023249 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fc7200076a0 con 0x7fc72806d300 2023-08-24T10:09:30.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.916+0000 7fc72eaac640 1 -- 172.21.15.60:0/3248023249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7281318e0 con 0x7fc72806d300 2023-08-24T10:09:30.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.916+0000 7fc7257fa640 1 --2- 172.21.15.60:0/3248023249 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc714037130 0x7fc714039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:30.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.917+0000 7fc7257fa640 1 -- 172.21.15.60:0/3248023249 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fc720046e90 con 0x7fc72806d300 2023-08-24T10:09:30.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.917+0000 7fc727fff640 1 --2- 172.21.15.60:0/3248023249 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc714037130 0x7fc714039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:30.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.917+0000 7fc727fff640 1 --2- 172.21.15.60:0/3248023249 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc714037130 0x7fc714039640 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:30.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:30.921+0000 7fc7257fa640 1 -- 172.21.15.60:0/3248023249 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fc7200094c0 con 0x7fc72806d300 2023-08-24T10:09:31.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.094+0000 7fc72eaac640 1 -- 172.21.15.60:0/3248023249 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc728097710 con 0x7fc714037130 2023-08-24T10:09:31.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.096+0000 7fc7257fa640 1 -- 172.21.15.60:0/3248023249 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7fc71c009ba0 con 0x7fc714037130 2023-08-24T10:09:31.097 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:31.098 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:31.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.098+0000 7fc72eaac640 1 -- 172.21.15.60:0/3248023249 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc714037130 msgr2=0x7fc714039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:31.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.098+0000 7fc72eaac640 1 --2- 172.21.15.60:0/3248023249 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc714037130 0x7fc714039640 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:31.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.098+0000 7fc72eaac640 1 -- 172.21.15.60:0/3248023249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc72806d300 msgr2=0x7fc728126450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:31.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.098+0000 7fc72eaac640 1 --2- 172.21.15.60:0/3248023249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc72806d300 0x7fc728126450 crc :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:31.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.098+0000 7fc72eaac640 1 -- 172.21.15.60:0/3248023249 shutdown_connections 2023-08-24T10:09:31.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.098+0000 7fc72eaac640 1 --2- 172.21.15.60:0/3248023249 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc714037130 0x7fc714039640 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:31.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.098+0000 7fc72eaac640 1 --2- 172.21.15.60:0/3248023249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc72806d300 0x7fc728126450 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:31.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.098+0000 7fc72eaac640 1 -- 172.21.15.60:0/3248023249 >> 172.21.15.60:0/3248023249 conn(0x7fc72809e110 msgr2=0x7fc7280964f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:31.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.098+0000 7fc72eaac640 1 -- 172.21.15.60:0/3248023249 shutdown_connections 2023-08-24T10:09:31.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.098+0000 7fc72eaac640 1 -- 172.21.15.60:0/3248023249 wait complete. 2023-08-24T10:09:31.109 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2023-08-24T10:09:29.547277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:31.110 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:09:31.111 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:31.111 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:09:31.111 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:31.111 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:09:31.112 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:31.112 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:09:31.112 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:31.113 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:09:31.113 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:31.113 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:09:31.113 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:31.113 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:09:31.114 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:31.114 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:09:31.114 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:31.114 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:09:31.115 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:31.115 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:09:31.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.222+0000 7f43ba31e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:31.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.225+0000 7f43ba31e640 1 Processor -- start 2023-08-24T10:09:31.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.225+0000 7f43ba31e640 1 -- start start 2023-08-24T10:09:31.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.226+0000 7f43ba31e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43b4135a60 0x7f43b4137e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:31.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.226+0000 7f43ba31e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b409d750 con 0x7f43b412f610 2023-08-24T10:09:31.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.226+0000 7f43ba31e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b4130130 con 0x7f43b4135a60 2023-08-24T10:09:31.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.226+0000 7f43b37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43b4135a60 0x7f43b4137e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.226+0000 7f43b37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43b4135a60 0x7f43b4137e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51818/0 (socket says 172.21.15.60:51818) 2023-08-24T10:09:31.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.226+0000 7f43b37fe640 1 -- 172.21.15.60:0/833649302 learned_addr learned my addr 172.21.15.60:0/833649302 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.227+0000 7f43b37fe640 1 -- 172.21.15.60:0/833649302 >> v1:172.21.15.60:6789/0 conn(0x7f43b412f610 legacy=0x7f43b412fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.227+0000 7f43b37fe640 1 -- 172.21.15.60:0/833649302 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b409d980 con 0x7f43b4135a60 2023-08-24T10:09:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.227+0000 7f43b37fe640 1 --2- 172.21.15.60:0/833649302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b4135a60 0x7f43b4137e80 crc :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4850cc95400cb2d8 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:31.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.227+0000 7f43b2ffd640 1 -- 172.21.15.60:0/833649302 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f439c009dc0 con 0x7f43b4135a60 2023-08-24T10:09:31.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.227+0000 7f43b2ffd640 1 -- 172.21.15.60:0/833649302 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f439c00e230 con 0x7f43b4135a60 2023-08-24T10:09:31.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.227+0000 7f43b2ffd640 1 -- 172.21.15.60:0/833649302 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f439c00e4d0 con 0x7f43b4135a60 2023-08-24T10:09:31.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.228+0000 7f43ba31e640 1 -- 172.21.15.60:0/833649302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b4135a60 msgr2=0x7f43b4137e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:31.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.228+0000 7f43ba31e640 1 --2- 172.21.15.60:0/833649302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b4135a60 0x7f43b4137e80 crc :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:31.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.228+0000 7f43ba31e640 1 -- 172.21.15.60:0/833649302 shutdown_connections 2023-08-24T10:09:31.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.228+0000 7f43ba31e640 1 --2- 172.21.15.60:0/833649302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b4135a60 0x7f43b4137e80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:31.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.228+0000 7f43ba31e640 1 -- 172.21.15.60:0/833649302 >> 172.21.15.60:0/833649302 conn(0x7f43b40a2500 msgr2=0x7f43b40a2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.228+0000 7f43ba31e640 1 -- 172.21.15.60:0/833649302 shutdown_connections 2023-08-24T10:09:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.228+0000 7f43ba31e640 1 -- 172.21.15.60:0/833649302 wait complete. 2023-08-24T10:09:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.228+0000 7f43ba31e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:31.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.229+0000 7f43ba31e640 1 Processor -- start 2023-08-24T10:09:31.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.229+0000 7f43ba31e640 1 -- start start 2023-08-24T10:09:31.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.230+0000 7f43ba31e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b412f610 0x7f43b40a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:31.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.230+0000 7f43ba31e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b40a08e0 con 0x7f43b412f610 2023-08-24T10:09:31.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.230+0000 7f43b3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b412f610 0x7f43b40a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:31.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.230+0000 7f43b3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b412f610 0x7f43b40a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51824/0 (socket says 172.21.15.60:51824) 2023-08-24T10:09:31.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.230+0000 7f43b3fff640 1 -- 172.21.15.60:0/1888328192 learned_addr learned my addr 172.21.15.60:0/1888328192 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:31.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.230+0000 7f43b3fff640 1 -- 172.21.15.60:0/1888328192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b411d420 con 0x7f43b412f610 2023-08-24T10:09:31.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.230+0000 7f43b3fff640 1 --2- 172.21.15.60:0/1888328192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b412f610 0x7f43b40a0d10 crc :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.230+0000 7f43b0ff9640 1 -- 172.21.15.60:0/1888328192 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f43a400c7e0 con 0x7f43b412f610 2023-08-24T10:09:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.230+0000 7f43b0ff9640 1 -- 172.21.15.60:0/1888328192 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f43a400c940 con 0x7f43b412f610 2023-08-24T10:09:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.230+0000 7f43ba31e640 1 -- 172.21.15.60:0/1888328192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b40a0540 con 0x7f43b412f610 2023-08-24T10:09:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.230+0000 7f43b0ff9640 1 -- 172.21.15.60:0/1888328192 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f43a400cbe0 con 0x7f43b412f610 2023-08-24T10:09:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.230+0000 7f43ba31e640 1 -- 172.21.15.60:0/1888328192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b411c810 con 0x7f43b412f610 2023-08-24T10:09:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.231+0000 7f43b0ff9640 1 -- 172.21.15.60:0/1888328192 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f43a4011310 con 0x7f43b412f610 2023-08-24T10:09:31.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.231+0000 7f43ba31e640 1 -- 172.21.15.60:0/1888328192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b40a0730 con 0x7f43b412f610 2023-08-24T10:09:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.231+0000 7f43b0ff9640 1 --2- 172.21.15.60:0/1888328192 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f43940370e0 0x7f43940395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.231+0000 7f43b0ff9640 1 -- 172.21.15.60:0/1888328192 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f43a4046bc0 con 0x7f43b412f610 2023-08-24T10:09:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.232+0000 7f43b37fe640 1 --2- 172.21.15.60:0/1888328192 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f43940370e0 0x7f43940395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:31.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.232+0000 7f43b37fe640 1 --2- 172.21.15.60:0/1888328192 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f43940370e0 0x7f43940395f0 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:31.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.236+0000 7f43b0ff9640 1 -- 172.21.15.60:0/1888328192 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f43a400ad00 con 0x7f43b412f610 2023-08-24T10:09:31.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.484+0000 7f43ba31e640 1 -- 172.21.15.60:0/1888328192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f43b4061800 con 0x7f43b412f610 2023-08-24T10:09:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.485+0000 7f43b0ff9640 1 -- 172.21.15.60:0/1888328192 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f43a400a910 con 0x7f43b412f610 2023-08-24T10:09:31.486 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:31.486 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":282,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:09:31.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.487+0000 7f43ba31e640 1 -- 172.21.15.60:0/1888328192 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f43940370e0 msgr2=0x7f43940395f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:31.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.487+0000 7f43ba31e640 1 --2- 172.21.15.60:0/1888328192 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f43940370e0 0x7f43940395f0 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:31.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.487+0000 7f43ba31e640 1 -- 172.21.15.60:0/1888328192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b412f610 msgr2=0x7f43b40a0d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:31.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.487+0000 7f43ba31e640 1 --2- 172.21.15.60:0/1888328192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b412f610 0x7f43b40a0d10 crc :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:31.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.487+0000 7f43ba31e640 1 -- 172.21.15.60:0/1888328192 shutdown_connections 2023-08-24T10:09:31.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.487+0000 7f43ba31e640 1 --2- 172.21.15.60:0/1888328192 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f43940370e0 0x7f43940395f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:31.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.487+0000 7f43ba31e640 1 --2- 172.21.15.60:0/1888328192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43b412f610 0x7f43b40a0d10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:31.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.487+0000 7f43ba31e640 1 -- 172.21.15.60:0/1888328192 >> 172.21.15.60:0/1888328192 conn(0x7f43b40a2500 msgr2=0x7f43b4099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:31.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.487+0000 7f43ba31e640 1 -- 172.21.15.60:0/1888328192 shutdown_connections 2023-08-24T10:09:31.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.487+0000 7f43ba31e640 1 -- 172.21.15.60:0/1888328192 wait complete. 2023-08-24T10:09:31.498 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:09:31.499 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:31.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.605+0000 7f7490aec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:31.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.609+0000 7f7490aec640 1 Processor -- start 2023-08-24T10:09:31.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.609+0000 7f7490aec640 1 -- start start 2023-08-24T10:09:31.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.610+0000 7f7490aec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f748c135980 0x7f748c137da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.610+0000 7f7490aec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748c132610 con 0x7f748c132b50 2023-08-24T10:09:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.610+0000 7f7490aec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748c09c260 con 0x7f748c135980 2023-08-24T10:09:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.610+0000 7f7489d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f748c135980 0x7f748c137da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.610+0000 7f7489d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f748c135980 0x7f748c137da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51832/0 (socket says 172.21.15.60:51832) 2023-08-24T10:09:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.610+0000 7f7489d9b640 1 -- 172.21.15.60:0/4294847129 learned_addr learned my addr 172.21.15.60:0/4294847129 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.610+0000 7f748a59c640 1 --1- 172.21.15.60:0/4294847129 >> v1:172.21.15.60:6789/0 conn(0x7f748c132b50 0x7f748c132f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38270/0 (socket says 172.21.15.60:38270) 2023-08-24T10:09:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.611+0000 7f7489d9b640 1 -- 172.21.15.60:0/4294847129 >> v1:172.21.15.60:6789/0 conn(0x7f748c132b50 legacy=0x7f748c132f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.611+0000 7f7489d9b640 1 -- 172.21.15.60:0/4294847129 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c132840 con 0x7f748c135980 2023-08-24T10:09:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.611+0000 7f7489d9b640 1 --2- 172.21.15.60:0/4294847129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f748c135980 0x7f748c137da0 crc :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a32d7e3f72d669d server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.611+0000 7f748959a640 1 -- 172.21.15.60:0/4294847129 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7474009dd0 con 0x7f748c135980 2023-08-24T10:09:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.611+0000 7f748959a640 1 -- 172.21.15.60:0/4294847129 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f747400e230 con 0x7f748c135980 2023-08-24T10:09:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.611+0000 7f748959a640 1 -- 172.21.15.60:0/4294847129 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f747400e4d0 con 0x7f748c135980 2023-08-24T10:09:31.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.611+0000 7f7490aec640 1 -- 172.21.15.60:0/4294847129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f748c135980 msgr2=0x7f748c137da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:31.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.611+0000 7f7490aec640 1 --2- 172.21.15.60:0/4294847129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f748c135980 0x7f748c137da0 crc :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.612+0000 7f7490aec640 1 -- 172.21.15.60:0/4294847129 shutdown_connections 2023-08-24T10:09:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.612+0000 7f7490aec640 1 --2- 172.21.15.60:0/4294847129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f748c135980 0x7f748c137da0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.612+0000 7f7490aec640 1 -- 172.21.15.60:0/4294847129 >> 172.21.15.60:0/4294847129 conn(0x7f748c0a26a0 msgr2=0x7f748c0a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.612+0000 7f7490aec640 1 -- 172.21.15.60:0/4294847129 shutdown_connections 2023-08-24T10:09:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.612+0000 7f7490aec640 1 -- 172.21.15.60:0/4294847129 wait complete. 2023-08-24T10:09:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.612+0000 7f7490aec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.613+0000 7f7490aec640 1 Processor -- start 2023-08-24T10:09:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.613+0000 7f7490aec640 1 -- start start 2023-08-24T10:09:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.614+0000 7f7490aec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f748c132b50 0x7f748c0a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.614+0000 7f7490aec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c0a1b60 con 0x7f748c132b50 2023-08-24T10:09:31.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.614+0000 7f748a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f748c132b50 0x7f748c0a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:31.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.614+0000 7f748a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f748c132b50 0x7f748c0a0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51838/0 (socket says 172.21.15.60:51838) 2023-08-24T10:09:31.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.614+0000 7f748a59c640 1 -- 172.21.15.60:0/1281560862 learned_addr learned my addr 172.21.15.60:0/1281560862 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.614+0000 7f748a59c640 1 -- 172.21.15.60:0/1281560862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c09fab0 con 0x7f748c132b50 2023-08-24T10:09:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.614+0000 7f748a59c640 1 --2- 172.21.15.60:0/1281560862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f748c132b50 0x7f748c0a0000 crc :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.614+0000 7f7482ffd640 1 -- 172.21.15.60:0/1281560862 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f74840071e0 con 0x7f748c132b50 2023-08-24T10:09:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.614+0000 7f7482ffd640 1 -- 172.21.15.60:0/1281560862 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7484007340 con 0x7f748c132b50 2023-08-24T10:09:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.614+0000 7f7490aec640 1 -- 172.21.15.60:0/1281560862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748c06a240 con 0x7f748c132b50 2023-08-24T10:09:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.614+0000 7f7482ffd640 1 -- 172.21.15.60:0/1281560862 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f74840075e0 con 0x7f748c132b50 2023-08-24T10:09:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.614+0000 7f7490aec640 1 -- 172.21.15.60:0/1281560862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748c11ac50 con 0x7f748c132b50 2023-08-24T10:09:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.615+0000 7f7482ffd640 1 -- 172.21.15.60:0/1281560862 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f74840057e0 con 0x7f748c132b50 2023-08-24T10:09:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.615+0000 7f7490aec640 1 -- 172.21.15.60:0/1281560862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748c061800 con 0x7f748c132b50 2023-08-24T10:09:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.615+0000 7f7482ffd640 1 --2- 172.21.15.60:0/1281560862 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f746c037130 0x7f746c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.615+0000 7f7482ffd640 1 -- 172.21.15.60:0/1281560862 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f7484046080 con 0x7f748c132b50 2023-08-24T10:09:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.616+0000 7f7489d9b640 1 --2- 172.21.15.60:0/1281560862 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f746c037130 0x7f746c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.616+0000 7f7489d9b640 1 --2- 172.21.15.60:0/1281560862 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f746c037130 0x7f746c039640 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:31.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.620+0000 7f7482ffd640 1 -- 172.21.15.60:0/1281560862 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f748400d720 con 0x7f748c132b50 2023-08-24T10:09:31.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.793+0000 7f7490aec640 1 -- 172.21.15.60:0/1281560862 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f748c09f940 con 0x7f746c037130 2023-08-24T10:09:31.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.795+0000 7f7482ffd640 1 -- 172.21.15.60:0/1281560862 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f748c09f940 con 0x7f746c037130 2023-08-24T10:09:31.796 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:31.796 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:31.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.797+0000 7f7490aec640 1 -- 172.21.15.60:0/1281560862 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f746c037130 msgr2=0x7f746c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:31.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.797+0000 7f7490aec640 1 --2- 172.21.15.60:0/1281560862 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f746c037130 0x7f746c039640 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:31.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.797+0000 7f7490aec640 1 -- 172.21.15.60:0/1281560862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f748c132b50 msgr2=0x7f748c0a0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:31.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.797+0000 7f7490aec640 1 --2- 172.21.15.60:0/1281560862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f748c132b50 0x7f748c0a0000 crc :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:31.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.797+0000 7f7490aec640 1 -- 172.21.15.60:0/1281560862 shutdown_connections 2023-08-24T10:09:31.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.797+0000 7f7490aec640 1 --2- 172.21.15.60:0/1281560862 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f746c037130 0x7f746c039640 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:31.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.797+0000 7f7490aec640 1 --2- 172.21.15.60:0/1281560862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f748c132b50 0x7f748c0a0000 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:31.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.797+0000 7f7490aec640 1 -- 172.21.15.60:0/1281560862 >> 172.21.15.60:0/1281560862 conn(0x7f748c0a26a0 msgr2=0x7f748c1baaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:31.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.797+0000 7f7490aec640 1 -- 172.21.15.60:0/1281560862 shutdown_connections 2023-08-24T10:09:31.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:31.797+0000 7f7490aec640 1 -- 172.21.15.60:0/1281560862 wait complete. 2023-08-24T10:09:31.808 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2023-08-24T10:09:31.547704+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:32.848 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~265s 2023-08-24T10:09:32.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~265s 2023-08-24T10:09:32.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~265s 2023-08-24T10:09:32.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~265s 2023-08-24T10:09:34.811 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:34.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.931+0000 7f757d4ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.935+0000 7f757d4ac640 1 Processor -- start 2023-08-24T10:09:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.935+0000 7f757d4ac640 1 -- start start 2023-08-24T10:09:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.936+0000 7f757d4ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7578071e40 0x7f7578096800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.936+0000 7f757d4ac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75780977f0 con 0x7f757806f100 2023-08-24T10:09:34.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.936+0000 7f757d4ac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757809a280 con 0x7f7578071e40 2023-08-24T10:09:34.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.936+0000 7f7576ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7578071e40 0x7f7578096800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:34.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.936+0000 7f7576ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7578071e40 0x7f7578096800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51844/0 (socket says 172.21.15.60:51844) 2023-08-24T10:09:34.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.936+0000 7f7576ffd640 1 -- 172.21.15.60:0/1556359601 learned_addr learned my addr 172.21.15.60:0/1556359601 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:34.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.936+0000 7f75767fc640 1 --1- 172.21.15.60:0/1556359601 >> v1:172.21.15.60:6789/0 conn(0x7f757806f100 0x7f757806f510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38272/0 (socket says 172.21.15.60:38272) 2023-08-24T10:09:34.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.936+0000 7f7576ffd640 1 -- 172.21.15.60:0/1556359601 >> v1:172.21.15.60:6789/0 conn(0x7f757806f100 legacy=0x7f757806f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:34.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.936+0000 7f7576ffd640 1 -- 172.21.15.60:0/1556359601 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757809a6b0 con 0x7f7578071e40 2023-08-24T10:09:34.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.936+0000 7f7576ffd640 1 --2- 172.21.15.60:0/1556359601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7578071e40 0x7f7578096800 crc :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b942581a8046976 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:34.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.937+0000 7f7575ffb640 1 -- 172.21.15.60:0/1556359601 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7568009dd0 con 0x7f7578071e40 2023-08-24T10:09:34.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.937+0000 7f7575ffb640 1 -- 172.21.15.60:0/1556359601 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f756800e230 con 0x7f7578071e40 2023-08-24T10:09:34.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.937+0000 7f7575ffb640 1 -- 172.21.15.60:0/1556359601 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f756800e4d0 con 0x7f7578071e40 2023-08-24T10:09:34.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.937+0000 7f757d4ac640 1 -- 172.21.15.60:0/1556359601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7578071e40 msgr2=0x7f7578096800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:34.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.937+0000 7f757d4ac640 1 --2- 172.21.15.60:0/1556359601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7578071e40 0x7f7578096800 crc :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:34.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.937+0000 7f757d4ac640 1 -- 172.21.15.60:0/1556359601 shutdown_connections 2023-08-24T10:09:34.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.937+0000 7f757d4ac640 1 --2- 172.21.15.60:0/1556359601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7578071e40 0x7f7578096800 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:34.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.937+0000 7f757d4ac640 1 -- 172.21.15.60:0/1556359601 >> 172.21.15.60:0/1556359601 conn(0x7f757809e110 msgr2=0x7f757809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:34.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.937+0000 7f757d4ac640 1 -- 172.21.15.60:0/1556359601 shutdown_connections 2023-08-24T10:09:34.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.937+0000 7f757d4ac640 1 -- 172.21.15.60:0/1556359601 wait complete. 2023-08-24T10:09:34.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.938+0000 7f757d4ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:34.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.939+0000 7f757d4ac640 1 Processor -- start 2023-08-24T10:09:34.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.939+0000 7f757d4ac640 1 -- start start 2023-08-24T10:09:34.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.939+0000 7f757d4ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f757806f100 0x7f75781c19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:34.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.939+0000 7f757d4ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757808ffe0 con 0x7f757806f100 2023-08-24T10:09:34.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.939+0000 7f7576ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f757806f100 0x7f75781c19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:34.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.939+0000 7f7576ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f757806f100 0x7f75781c19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51850/0 (socket says 172.21.15.60:51850) 2023-08-24T10:09:34.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.940+0000 7f7576ffd640 1 -- 172.21.15.60:0/237757753 learned_addr learned my addr 172.21.15.60:0/237757753 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:34.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.940+0000 7f7576ffd640 1 -- 172.21.15.60:0/237757753 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75781c14b0 con 0x7f757806f100 2023-08-24T10:09:34.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.940+0000 7f7576ffd640 1 --2- 172.21.15.60:0/237757753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f757806f100 0x7f75781c19f0 crc :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:34.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.940+0000 7f755f7fe640 1 -- 172.21.15.60:0/237757753 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7568014540 con 0x7f757806f100 2023-08-24T10:09:34.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.940+0000 7f755f7fe640 1 -- 172.21.15.60:0/237757753 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f75680146a0 con 0x7f757806f100 2023-08-24T10:09:34.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.940+0000 7f755f7fe640 1 -- 172.21.15.60:0/237757753 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7568014940 con 0x7f757806f100 2023-08-24T10:09:34.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.940+0000 7f757d4ac640 1 -- 172.21.15.60:0/237757753 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7578090720 con 0x7f757806f100 2023-08-24T10:09:34.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.940+0000 7f757d4ac640 1 -- 172.21.15.60:0/237757753 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7578092c10 con 0x7f757806f100 2023-08-24T10:09:34.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.941+0000 7f755f7fe640 1 -- 172.21.15.60:0/237757753 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f7568014af0 con 0x7f757806f100 2023-08-24T10:09:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.941+0000 7f755f7fe640 1 --2- 172.21.15.60:0/237757753 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7564037130 0x7f7564039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.941+0000 7f757d4ac640 1 -- 172.21.15.60:0/237757753 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7578061800 con 0x7f757806f100 2023-08-24T10:09:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.941+0000 7f755f7fe640 1 -- 172.21.15.60:0/237757753 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f7568019040 con 0x7f757806f100 2023-08-24T10:09:34.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.942+0000 7f75767fc640 1 --2- 172.21.15.60:0/237757753 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7564037130 0x7f7564039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:34.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.942+0000 7f75767fc640 1 --2- 172.21.15.60:0/237757753 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7564037130 0x7f7564039640 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:34.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:34.946+0000 7f755f7fe640 1 -- 172.21.15.60:0/237757753 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f7568018190 con 0x7f757806f100 2023-08-24T10:09:35.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.118+0000 7f757d4ac640 1 -- 172.21.15.60:0/237757753 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7578097a80 con 0x7f7564037130 2023-08-24T10:09:35.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.119+0000 7f755f7fe640 1 -- 172.21.15.60:0/237757753 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f756c0070b0 con 0x7f7564037130 2023-08-24T10:09:35.121 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:35.122 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:35.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.121+0000 7f757d4ac640 1 -- 172.21.15.60:0/237757753 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7564037130 msgr2=0x7f7564039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:35.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.121+0000 7f757d4ac640 1 --2- 172.21.15.60:0/237757753 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7564037130 0x7f7564039640 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:35.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.121+0000 7f757d4ac640 1 -- 172.21.15.60:0/237757753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f757806f100 msgr2=0x7f75781c19f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:35.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.121+0000 7f757d4ac640 1 --2- 172.21.15.60:0/237757753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f757806f100 0x7f75781c19f0 crc :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:35.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.121+0000 7f757d4ac640 1 -- 172.21.15.60:0/237757753 shutdown_connections 2023-08-24T10:09:35.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.121+0000 7f757d4ac640 1 --2- 172.21.15.60:0/237757753 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7564037130 0x7f7564039640 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:35.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.121+0000 7f757d4ac640 1 --2- 172.21.15.60:0/237757753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f757806f100 0x7f75781c19f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:35.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.121+0000 7f757d4ac640 1 -- 172.21.15.60:0/237757753 >> 172.21.15.60:0/237757753 conn(0x7f757809e110 msgr2=0x7f75780985b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:35.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.121+0000 7f757d4ac640 1 -- 172.21.15.60:0/237757753 shutdown_connections 2023-08-24T10:09:35.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.121+0000 7f757d4ac640 1 -- 172.21.15.60:0/237757753 wait complete. 2023-08-24T10:09:35.133 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2023-08-24T10:09:33.548068+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:35.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:35.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.242+0000 7f5e3e51e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:35.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.246+0000 7f5e3e51e640 1 Processor -- start 2023-08-24T10:09:35.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.246+0000 7f5e3e51e640 1 -- start start 2023-08-24T10:09:35.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.247+0000 7f5e3e51e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5e38135980 0x7f5e38137da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:35.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.247+0000 7f5e3e51e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e38132610 con 0x7f5e38132b50 2023-08-24T10:09:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.247+0000 7f5e3e51e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3809c260 con 0x7f5e38135980 2023-08-24T10:09:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.247+0000 7f5e377fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5e38135980 0x7f5e38137da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.247+0000 7f5e377fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5e38135980 0x7f5e38137da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51856/0 (socket says 172.21.15.60:51856) 2023-08-24T10:09:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.247+0000 7f5e377fe640 1 -- 172.21.15.60:0/1985249549 learned_addr learned my addr 172.21.15.60:0/1985249549 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.247+0000 7f5e37fff640 1 --1- 172.21.15.60:0/1985249549 >> v1:172.21.15.60:6789/0 conn(0x7f5e38132b50 0x7f5e38132f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38282/0 (socket says 172.21.15.60:38282) 2023-08-24T10:09:35.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.248+0000 7f5e377fe640 1 -- 172.21.15.60:0/1985249549 >> v1:172.21.15.60:6789/0 conn(0x7f5e38132b50 legacy=0x7f5e38132f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.248+0000 7f5e377fe640 1 -- 172.21.15.60:0/1985249549 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e38132840 con 0x7f5e38135980 2023-08-24T10:09:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.248+0000 7f5e377fe640 1 --2- 172.21.15.60:0/1985249549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e38135980 0x7f5e38137da0 crc :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc2f244c855c243 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.248+0000 7f5e36ffd640 1 -- 172.21.15.60:0/1985249549 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5e2c009dd0 con 0x7f5e38135980 2023-08-24T10:09:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.248+0000 7f5e36ffd640 1 -- 172.21.15.60:0/1985249549 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5e2c00e230 con 0x7f5e38135980 2023-08-24T10:09:35.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.248+0000 7f5e36ffd640 1 -- 172.21.15.60:0/1985249549 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5e2c00e4d0 con 0x7f5e38135980 2023-08-24T10:09:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.249+0000 7f5e3e51e640 1 -- 172.21.15.60:0/1985249549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e38135980 msgr2=0x7f5e38137da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.249+0000 7f5e3e51e640 1 --2- 172.21.15.60:0/1985249549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e38135980 0x7f5e38137da0 crc :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.249+0000 7f5e3e51e640 1 -- 172.21.15.60:0/1985249549 shutdown_connections 2023-08-24T10:09:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.249+0000 7f5e3e51e640 1 --2- 172.21.15.60:0/1985249549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e38135980 0x7f5e38137da0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.249+0000 7f5e3e51e640 1 -- 172.21.15.60:0/1985249549 >> 172.21.15.60:0/1985249549 conn(0x7f5e380a26a0 msgr2=0x7f5e380a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.249+0000 7f5e3e51e640 1 -- 172.21.15.60:0/1985249549 shutdown_connections 2023-08-24T10:09:35.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.249+0000 7f5e3e51e640 1 -- 172.21.15.60:0/1985249549 wait complete. 2023-08-24T10:09:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.249+0000 7f5e3e51e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.250+0000 7f5e3e51e640 1 Processor -- start 2023-08-24T10:09:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.250+0000 7f5e3e51e640 1 -- start start 2023-08-24T10:09:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.251+0000 7f5e3e51e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e38132b50 0x7f5e380a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.251+0000 7f5e3e51e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e380a1b60 con 0x7f5e38132b50 2023-08-24T10:09:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.251+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e38132b50 0x7f5e380a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.251+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e38132b50 0x7f5e380a0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51866/0 (socket says 172.21.15.60:51866) 2023-08-24T10:09:35.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.251+0000 7f5e37fff640 1 -- 172.21.15.60:0/421300130 learned_addr learned my addr 172.21.15.60:0/421300130 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:35.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.251+0000 7f5e37fff640 1 -- 172.21.15.60:0/421300130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3809fab0 con 0x7f5e38132b50 2023-08-24T10:09:35.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.251+0000 7f5e37fff640 1 --2- 172.21.15.60:0/421300130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e38132b50 0x7f5e380a0000 crc :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:35.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.251+0000 7f5e34ff9640 1 -- 172.21.15.60:0/421300130 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5e280071e0 con 0x7f5e38132b50 2023-08-24T10:09:35.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.251+0000 7f5e34ff9640 1 -- 172.21.15.60:0/421300130 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5e28007340 con 0x7f5e38132b50 2023-08-24T10:09:35.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.251+0000 7f5e3e51e640 1 -- 172.21.15.60:0/421300130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3806a240 con 0x7f5e38132b50 2023-08-24T10:09:35.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.251+0000 7f5e34ff9640 1 -- 172.21.15.60:0/421300130 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5e280075e0 con 0x7f5e38132b50 2023-08-24T10:09:35.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.251+0000 7f5e3e51e640 1 -- 172.21.15.60:0/421300130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3811ac50 con 0x7f5e38132b50 2023-08-24T10:09:35.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.252+0000 7f5e34ff9640 1 -- 172.21.15.60:0/421300130 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f5e280057e0 con 0x7f5e38132b50 2023-08-24T10:09:35.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.252+0000 7f5e3e51e640 1 -- 172.21.15.60:0/421300130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e38061800 con 0x7f5e38132b50 2023-08-24T10:09:35.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.252+0000 7f5e34ff9640 1 --2- 172.21.15.60:0/421300130 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5e24037180 0x7f5e24039690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:35.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.253+0000 7f5e34ff9640 1 -- 172.21.15.60:0/421300130 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f5e28046140 con 0x7f5e38132b50 2023-08-24T10:09:35.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.253+0000 7f5e377fe640 1 --2- 172.21.15.60:0/421300130 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5e24037180 0x7f5e24039690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:35.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.253+0000 7f5e377fe640 1 --2- 172.21.15.60:0/421300130 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5e24037180 0x7f5e24039690 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:35.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.257+0000 7f5e34ff9640 1 -- 172.21.15.60:0/421300130 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f5e2800d720 con 0x7f5e38132b50 2023-08-24T10:09:35.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.431+0000 7f5e3e51e640 1 -- 172.21.15.60:0/421300130 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5e3809f940 con 0x7f5e24037180 2023-08-24T10:09:35.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.433+0000 7f5e34ff9640 1 -- 172.21.15.60:0/421300130 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f5e3809f940 con 0x7f5e24037180 2023-08-24T10:09:35.435 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.435+0000 7f5e3e51e640 1 -- 172.21.15.60:0/421300130 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5e24037180 msgr2=0x7f5e24039690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.435+0000 7f5e3e51e640 1 --2- 172.21.15.60:0/421300130 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5e24037180 0x7f5e24039690 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.435+0000 7f5e3e51e640 1 -- 172.21.15.60:0/421300130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e38132b50 msgr2=0x7f5e380a0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.435+0000 7f5e3e51e640 1 --2- 172.21.15.60:0/421300130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e38132b50 0x7f5e380a0000 crc :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.435+0000 7f5e3e51e640 1 -- 172.21.15.60:0/421300130 shutdown_connections 2023-08-24T10:09:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.435+0000 7f5e3e51e640 1 --2- 172.21.15.60:0/421300130 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5e24037180 0x7f5e24039690 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.435+0000 7f5e3e51e640 1 --2- 172.21.15.60:0/421300130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e38132b50 0x7f5e380a0000 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.435+0000 7f5e3e51e640 1 -- 172.21.15.60:0/421300130 >> 172.21.15.60:0/421300130 conn(0x7f5e380a26a0 msgr2=0x7f5e381baaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.435+0000 7f5e3e51e640 1 -- 172.21.15.60:0/421300130 shutdown_connections 2023-08-24T10:09:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.435+0000 7f5e3e51e640 1 -- 172.21.15.60:0/421300130 wait complete. 2023-08-24T10:09:35.447 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2023-08-24T10:09:33.548068+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:35.448 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:09:35.448 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:35.449 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:09:35.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:35.449 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:09:35.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:35.450 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:09:35.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:35.450 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:09:35.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:35.451 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:09:35.451 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:35.451 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:09:35.451 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:35.452 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:09:35.452 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:35.452 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:09:35.452 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:35.453 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:09:35.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.559+0000 7f9c42656640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:35.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.562+0000 7f9c42656640 1 Processor -- start 2023-08-24T10:09:35.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.563+0000 7f9c42656640 1 -- start start 2023-08-24T10:09:35.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.563+0000 7f9c42656640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c3c135a60 0x7f9c3c137e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:35.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.563+0000 7f9c42656640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3c09d750 con 0x7f9c3c12f610 2023-08-24T10:09:35.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.563+0000 7f9c42656640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3c130130 con 0x7f9c3c135a60 2023-08-24T10:09:35.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.563+0000 7f9c3b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c3c135a60 0x7f9c3c137e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:35.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.563+0000 7f9c3b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c3c135a60 0x7f9c3c137e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51882/0 (socket says 172.21.15.60:51882) 2023-08-24T10:09:35.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.564+0000 7f9c3b7fe640 1 -- 172.21.15.60:0/2273561456 learned_addr learned my addr 172.21.15.60:0/2273561456 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:35.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.564+0000 7f9c3bfff640 1 --1- 172.21.15.60:0/2273561456 >> v1:172.21.15.60:6789/0 conn(0x7f9c3c12f610 0x7f9c3c12fa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38286/0 (socket says 172.21.15.60:38286) 2023-08-24T10:09:35.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.564+0000 7f9c3b7fe640 1 -- 172.21.15.60:0/2273561456 >> v1:172.21.15.60:6789/0 conn(0x7f9c3c12f610 legacy=0x7f9c3c12fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:35.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.564+0000 7f9c3b7fe640 1 -- 172.21.15.60:0/2273561456 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c09d980 con 0x7f9c3c135a60 2023-08-24T10:09:35.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.564+0000 7f9c3b7fe640 1 --2- 172.21.15.60:0/2273561456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c3c135a60 0x7f9c3c137e80 crc :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad8752f468bfb14 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:35.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.564+0000 7f9c3affd640 1 -- 172.21.15.60:0/2273561456 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9c30009dc0 con 0x7f9c3c135a60 2023-08-24T10:09:35.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.564+0000 7f9c3affd640 1 -- 172.21.15.60:0/2273561456 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f9c3000e230 con 0x7f9c3c135a60 2023-08-24T10:09:35.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.564+0000 7f9c3affd640 1 -- 172.21.15.60:0/2273561456 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9c3000e4d0 con 0x7f9c3c135a60 2023-08-24T10:09:35.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.565+0000 7f9c42656640 1 -- 172.21.15.60:0/2273561456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c3c135a60 msgr2=0x7f9c3c137e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:35.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.565+0000 7f9c42656640 1 --2- 172.21.15.60:0/2273561456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c3c135a60 0x7f9c3c137e80 crc :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:35.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.565+0000 7f9c42656640 1 -- 172.21.15.60:0/2273561456 shutdown_connections 2023-08-24T10:09:35.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.565+0000 7f9c42656640 1 --2- 172.21.15.60:0/2273561456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c3c135a60 0x7f9c3c137e80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:35.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.565+0000 7f9c42656640 1 -- 172.21.15.60:0/2273561456 >> 172.21.15.60:0/2273561456 conn(0x7f9c3c0a2500 msgr2=0x7f9c3c0a2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:35.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.565+0000 7f9c42656640 1 -- 172.21.15.60:0/2273561456 shutdown_connections 2023-08-24T10:09:35.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.565+0000 7f9c42656640 1 -- 172.21.15.60:0/2273561456 wait complete. 2023-08-24T10:09:35.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.565+0000 7f9c42656640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:35.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.566+0000 7f9c42656640 1 Processor -- start 2023-08-24T10:09:35.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.566+0000 7f9c42656640 1 -- start start 2023-08-24T10:09:35.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.567+0000 7f9c42656640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c3c12f610 0x7f9c3c0a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:35.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.567+0000 7f9c42656640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0a08e0 con 0x7f9c3c12f610 2023-08-24T10:09:35.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.567+0000 7f9c3bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c3c12f610 0x7f9c3c0a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:35.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.567+0000 7f9c3bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c3c12f610 0x7f9c3c0a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51896/0 (socket says 172.21.15.60:51896) 2023-08-24T10:09:35.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.567+0000 7f9c3bfff640 1 -- 172.21.15.60:0/2484657980 learned_addr learned my addr 172.21.15.60:0/2484657980 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:35.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.567+0000 7f9c3bfff640 1 -- 172.21.15.60:0/2484657980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c11d420 con 0x7f9c3c12f610 2023-08-24T10:09:35.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.567+0000 7f9c3bfff640 1 --2- 172.21.15.60:0/2484657980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c3c12f610 0x7f9c3c0a0d10 crc :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:35.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.567+0000 7f9c38ff9640 1 -- 172.21.15.60:0/2484657980 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9c2c0071e0 con 0x7f9c3c12f610 2023-08-24T10:09:35.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.568+0000 7f9c38ff9640 1 -- 172.21.15.60:0/2484657980 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f9c2c007340 con 0x7f9c3c12f610 2023-08-24T10:09:35.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.568+0000 7f9c38ff9640 1 -- 172.21.15.60:0/2484657980 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9c2c0075e0 con 0x7f9c3c12f610 2023-08-24T10:09:35.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.568+0000 7f9c42656640 1 -- 172.21.15.60:0/2484657980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3c0a0540 con 0x7f9c3c12f610 2023-08-24T10:09:35.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.568+0000 7f9c42656640 1 -- 172.21.15.60:0/2484657980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3c11c810 con 0x7f9c3c12f610 2023-08-24T10:09:35.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.568+0000 7f9c38ff9640 1 -- 172.21.15.60:0/2484657980 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f9c2c0057e0 con 0x7f9c3c12f610 2023-08-24T10:09:35.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.569+0000 7f9c38ff9640 1 --2- 172.21.15.60:0/2484657980 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9c28037130 0x7f9c28039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:35.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.569+0000 7f9c38ff9640 1 -- 172.21.15.60:0/2484657980 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f9c2c046470 con 0x7f9c3c12f610 2023-08-24T10:09:35.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.569+0000 7f9c42656640 1 -- 172.21.15.60:0/2484657980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3c061800 con 0x7f9c3c12f610 2023-08-24T10:09:35.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.569+0000 7f9c3b7fe640 1 --2- 172.21.15.60:0/2484657980 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9c28037130 0x7f9c28039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:35.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.569+0000 7f9c3b7fe640 1 --2- 172.21.15.60:0/2484657980 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9c28037130 0x7f9c28039640 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:35.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.573+0000 7f9c38ff9640 1 -- 172.21.15.60:0/2484657980 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f9c2c00d330 con 0x7f9c3c12f610 2023-08-24T10:09:35.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.822+0000 7f9c42656640 1 -- 172.21.15.60:0/2484657980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9c3c0a06d0 con 0x7f9c3c12f610 2023-08-24T10:09:35.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.823+0000 7f9c38ff9640 1 -- 172.21.15.60:0/2484657980 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f9c2c00d150 con 0x7f9c3c12f610 2023-08-24T10:09:35.824 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:35.824 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":286,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:09:35.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.824+0000 7f9c42656640 1 -- 172.21.15.60:0/2484657980 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9c28037130 msgr2=0x7f9c28039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:35.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.824+0000 7f9c42656640 1 --2- 172.21.15.60:0/2484657980 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9c28037130 0x7f9c28039640 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:35.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.824+0000 7f9c42656640 1 -- 172.21.15.60:0/2484657980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c3c12f610 msgr2=0x7f9c3c0a0d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:35.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.824+0000 7f9c42656640 1 --2- 172.21.15.60:0/2484657980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c3c12f610 0x7f9c3c0a0d10 crc :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:35.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.824+0000 7f9c42656640 1 -- 172.21.15.60:0/2484657980 shutdown_connections 2023-08-24T10:09:35.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.824+0000 7f9c42656640 1 --2- 172.21.15.60:0/2484657980 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9c28037130 0x7f9c28039640 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:35.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.824+0000 7f9c42656640 1 --2- 172.21.15.60:0/2484657980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c3c12f610 0x7f9c3c0a0d10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:35.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.824+0000 7f9c42656640 1 -- 172.21.15.60:0/2484657980 >> 172.21.15.60:0/2484657980 conn(0x7f9c3c0a2500 msgr2=0x7f9c3c099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:35.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.825+0000 7f9c42656640 1 -- 172.21.15.60:0/2484657980 shutdown_connections 2023-08-24T10:09:35.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.825+0000 7f9c42656640 1 -- 172.21.15.60:0/2484657980 wait complete. 2023-08-24T10:09:35.837 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:09:35.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:35.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.945+0000 7fa6c239e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:35.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.949+0000 7fa6c239e640 1 Processor -- start 2023-08-24T10:09:35.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.949+0000 7fa6c239e640 1 -- start start 2023-08-24T10:09:35.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.950+0000 7fa6c239e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa6bc071e40 0x7fa6bc096800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:35.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.950+0000 7fa6c239e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6bc0977f0 con 0x7fa6bc06f100 2023-08-24T10:09:35.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.950+0000 7fa6c239e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6bc09a280 con 0x7fa6bc071e40 2023-08-24T10:09:35.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.950+0000 7fa6bbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa6bc071e40 0x7fa6bc096800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:35.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.950+0000 7fa6bbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa6bc071e40 0x7fa6bc096800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51898/0 (socket says 172.21.15.60:51898) 2023-08-24T10:09:35.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.950+0000 7fa6bb7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa6bc06f100 0x7fa6bc06f510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38298/0 (socket says 172.21.15.60:38298) 2023-08-24T10:09:35.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.950+0000 7fa6bbfff640 1 -- 172.21.15.60:0/345817836 learned_addr learned my addr 172.21.15.60:0/345817836 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:35.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.950+0000 7fa6bbfff640 1 -- 172.21.15.60:0/345817836 >> v1:172.21.15.60:6789/0 conn(0x7fa6bc06f100 legacy=0x7fa6bc06f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:35.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.950+0000 7fa6bbfff640 1 -- 172.21.15.60:0/345817836 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc09a6b0 con 0x7fa6bc071e40 2023-08-24T10:09:35.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.950+0000 7fa6bbfff640 1 --2- 172.21.15.60:0/345817836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6bc071e40 0x7fa6bc096800 crc :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2b95ebde5383e7 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:35.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.950+0000 7fa6baffd640 1 -- 172.21.15.60:0/345817836 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa6ac009dd0 con 0x7fa6bc071e40 2023-08-24T10:09:35.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.950+0000 7fa6baffd640 1 -- 172.21.15.60:0/345817836 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa6ac00e230 con 0x7fa6bc071e40 2023-08-24T10:09:35.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.950+0000 7fa6baffd640 1 -- 172.21.15.60:0/345817836 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa6ac00e4d0 con 0x7fa6bc071e40 2023-08-24T10:09:35.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.951+0000 7fa6c239e640 1 -- 172.21.15.60:0/345817836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6bc071e40 msgr2=0x7fa6bc096800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:35.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.951+0000 7fa6c239e640 1 --2- 172.21.15.60:0/345817836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6bc071e40 0x7fa6bc096800 crc :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:35.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.951+0000 7fa6c239e640 1 -- 172.21.15.60:0/345817836 shutdown_connections 2023-08-24T10:09:35.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.951+0000 7fa6c239e640 1 --2- 172.21.15.60:0/345817836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6bc071e40 0x7fa6bc096800 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:35.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.951+0000 7fa6c239e640 1 -- 172.21.15.60:0/345817836 >> 172.21.15.60:0/345817836 conn(0x7fa6bc09e110 msgr2=0x7fa6bc09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:35.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.951+0000 7fa6c239e640 1 -- 172.21.15.60:0/345817836 shutdown_connections 2023-08-24T10:09:35.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.951+0000 7fa6c239e640 1 -- 172.21.15.60:0/345817836 wait complete. 2023-08-24T10:09:35.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.951+0000 7fa6c239e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:35.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.952+0000 7fa6c239e640 1 Processor -- start 2023-08-24T10:09:35.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.952+0000 7fa6c239e640 1 -- start start 2023-08-24T10:09:35.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.953+0000 7fa6c239e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6bc06f100 0x7fa6bc1c19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:35.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.953+0000 7fa6c239e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc08ffe0 con 0x7fa6bc06f100 2023-08-24T10:09:35.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.953+0000 7fa6bbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6bc06f100 0x7fa6bc1c19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:35.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.953+0000 7fa6bbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6bc06f100 0x7fa6bc1c19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51904/0 (socket says 172.21.15.60:51904) 2023-08-24T10:09:35.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.953+0000 7fa6bbfff640 1 -- 172.21.15.60:0/2319090403 learned_addr learned my addr 172.21.15.60:0/2319090403 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:35.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.953+0000 7fa6bbfff640 1 -- 172.21.15.60:0/2319090403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc1c14b0 con 0x7fa6bc06f100 2023-08-24T10:09:35.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.954+0000 7fa6bbfff640 1 --2- 172.21.15.60:0/2319090403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6bc06f100 0x7fa6bc1c19f0 crc :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:35.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.954+0000 7fa6b8ff9640 1 -- 172.21.15.60:0/2319090403 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa6ac014540 con 0x7fa6bc06f100 2023-08-24T10:09:35.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.954+0000 7fa6b8ff9640 1 -- 172.21.15.60:0/2319090403 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa6ac0146a0 con 0x7fa6bc06f100 2023-08-24T10:09:35.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.954+0000 7fa6b8ff9640 1 -- 172.21.15.60:0/2319090403 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa6ac014940 con 0x7fa6bc06f100 2023-08-24T10:09:35.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.954+0000 7fa6c239e640 1 -- 172.21.15.60:0/2319090403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6bc090720 con 0x7fa6bc06f100 2023-08-24T10:09:35.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.954+0000 7fa6c239e640 1 -- 172.21.15.60:0/2319090403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6bc092c10 con 0x7fa6bc06f100 2023-08-24T10:09:35.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.955+0000 7fa6b8ff9640 1 -- 172.21.15.60:0/2319090403 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fa6ac014af0 con 0x7fa6bc06f100 2023-08-24T10:09:35.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.955+0000 7fa6c239e640 1 -- 172.21.15.60:0/2319090403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6bc061800 con 0x7fa6bc06f100 2023-08-24T10:09:35.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.955+0000 7fa6b8ff9640 1 --2- 172.21.15.60:0/2319090403 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa6a8037130 0x7fa6a8039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:35.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.955+0000 7fa6b8ff9640 1 -- 172.21.15.60:0/2319090403 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fa6ac019040 con 0x7fa6bc06f100 2023-08-24T10:09:35.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.955+0000 7fa6bb7fe640 1 --2- 172.21.15.60:0/2319090403 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa6a8037130 0x7fa6a8039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:35.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.956+0000 7fa6bb7fe640 1 --2- 172.21.15.60:0/2319090403 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa6a8037130 0x7fa6a8039640 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:35.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:35.960+0000 7fa6b8ff9640 1 -- 172.21.15.60:0/2319090403 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fa6ac018190 con 0x7fa6bc06f100 2023-08-24T10:09:36.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:36.130+0000 7fa6c239e640 1 -- 172.21.15.60:0/2319090403 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6bc097a80 con 0x7fa6a8037130 2023-08-24T10:09:36.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:36.132+0000 7fa6b8ff9640 1 -- 172.21.15.60:0/2319090403 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7fa6b00070b0 con 0x7fa6a8037130 2023-08-24T10:09:36.133 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:36.133 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:36.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:36.134+0000 7fa6c239e640 1 -- 172.21.15.60:0/2319090403 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa6a8037130 msgr2=0x7fa6a8039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:36.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:36.134+0000 7fa6c239e640 1 --2- 172.21.15.60:0/2319090403 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa6a8037130 0x7fa6a8039640 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:36.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:36.134+0000 7fa6c239e640 1 -- 172.21.15.60:0/2319090403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6bc06f100 msgr2=0x7fa6bc1c19f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:36.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:36.134+0000 7fa6c239e640 1 --2- 172.21.15.60:0/2319090403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6bc06f100 0x7fa6bc1c19f0 crc :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:36.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:36.134+0000 7fa6c239e640 1 -- 172.21.15.60:0/2319090403 shutdown_connections 2023-08-24T10:09:36.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:36.134+0000 7fa6c239e640 1 --2- 172.21.15.60:0/2319090403 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fa6a8037130 0x7fa6a8039640 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:36.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:36.134+0000 7fa6c239e640 1 --2- 172.21.15.60:0/2319090403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6bc06f100 0x7fa6bc1c19f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:36.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:36.134+0000 7fa6c239e640 1 -- 172.21.15.60:0/2319090403 >> 172.21.15.60:0/2319090403 conn(0x7fa6bc09e110 msgr2=0x7fa6bc0985b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:36.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:36.134+0000 7fa6c239e640 1 -- 172.21.15.60:0/2319090403 shutdown_connections 2023-08-24T10:09:36.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:36.134+0000 7fa6c239e640 1 -- 172.21.15.60:0/2319090403 wait complete. 2023-08-24T10:09:36.145 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2023-08-24T10:09:35.548531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:38.051 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~271s 2023-08-24T10:09:38.052 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~271s 2023-08-24T10:09:38.052 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~271s 2023-08-24T10:09:38.053 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~271s 2023-08-24T10:09:39.147 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:39.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.268+0000 7ff0c1aac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:39.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.272+0000 7ff0c1aac640 1 Processor -- start 2023-08-24T10:09:39.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.272+0000 7ff0c1aac640 1 -- start start 2023-08-24T10:09:39.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.273+0000 7ff0c1aac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff0bc131d40 0x7ff0bc134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:39.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.273+0000 7ff0c1aac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0bc099440 con 0x7ff0bc06d300 2023-08-24T10:09:39.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.273+0000 7ff0c1aac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0bc09a050 con 0x7ff0bc131d40 2023-08-24T10:09:39.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.273+0000 7ff0baffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff0bc131d40 0x7ff0bc134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:39.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.273+0000 7ff0baffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff0bc131d40 0x7ff0bc134160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37826/0 (socket says 172.21.15.60:37826) 2023-08-24T10:09:39.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.273+0000 7ff0baffd640 1 -- 172.21.15.60:0/3372161807 learned_addr learned my addr 172.21.15.60:0/3372161807 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:39.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.273+0000 7ff0baffd640 1 -- 172.21.15.60:0/3372161807 >> v1:172.21.15.60:6789/0 conn(0x7ff0bc06d300 legacy=0x7ff0bc12bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:39.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.273+0000 7ff0baffd640 1 -- 172.21.15.60:0/3372161807 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc12b9d0 con 0x7ff0bc131d40 2023-08-24T10:09:39.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.274+0000 7ff0baffd640 1 --2- 172.21.15.60:0/3372161807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0bc131d40 0x7ff0bc134160 crc :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6717e16685404d1f server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:39.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.274+0000 7ff0ba7fc640 1 -- 172.21.15.60:0/3372161807 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff0b0009dc0 con 0x7ff0bc131d40 2023-08-24T10:09:39.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.274+0000 7ff0ba7fc640 1 -- 172.21.15.60:0/3372161807 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ff0b000e230 con 0x7ff0bc131d40 2023-08-24T10:09:39.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.274+0000 7ff0ba7fc640 1 -- 172.21.15.60:0/3372161807 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff0b000e4d0 con 0x7ff0bc131d40 2023-08-24T10:09:39.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.274+0000 7ff0c1aac640 1 -- 172.21.15.60:0/3372161807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0bc131d40 msgr2=0x7ff0bc134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:39.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.274+0000 7ff0c1aac640 1 --2- 172.21.15.60:0/3372161807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0bc131d40 0x7ff0bc134160 crc :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:39.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.275+0000 7ff0c1aac640 1 -- 172.21.15.60:0/3372161807 shutdown_connections 2023-08-24T10:09:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.275+0000 7ff0c1aac640 1 --2- 172.21.15.60:0/3372161807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0bc131d40 0x7ff0bc134160 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.275+0000 7ff0c1aac640 1 -- 172.21.15.60:0/3372161807 >> 172.21.15.60:0/3372161807 conn(0x7ff0bc09e110 msgr2=0x7ff0bc09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.275+0000 7ff0c1aac640 1 -- 172.21.15.60:0/3372161807 shutdown_connections 2023-08-24T10:09:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.275+0000 7ff0c1aac640 1 -- 172.21.15.60:0/3372161807 wait complete. 2023-08-24T10:09:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.275+0000 7ff0c1aac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.276+0000 7ff0c1aac640 1 Processor -- start 2023-08-24T10:09:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.276+0000 7ff0c1aac640 1 -- start start 2023-08-24T10:09:39.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.277+0000 7ff0c1aac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0bc06d300 0x7ff0bc1356b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:39.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.277+0000 7ff0c1aac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc127be0 con 0x7ff0bc06d300 2023-08-24T10:09:39.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.277+0000 7ff0bb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0bc06d300 0x7ff0bc1356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:39.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.277+0000 7ff0bb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0bc06d300 0x7ff0bc1356b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37836/0 (socket says 172.21.15.60:37836) 2023-08-24T10:09:39.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.277+0000 7ff0bb7fe640 1 -- 172.21.15.60:0/3398039189 learned_addr learned my addr 172.21.15.60:0/3398039189 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:39.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.277+0000 7ff0bb7fe640 1 -- 172.21.15.60:0/3398039189 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc071c80 con 0x7ff0bc06d300 2023-08-24T10:09:39.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.277+0000 7ff0bb7fe640 1 --2- 172.21.15.60:0/3398039189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0bc06d300 0x7ff0bc1356b0 crc :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:39.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.277+0000 7ff0a3fff640 1 -- 172.21.15.60:0/3398039189 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff0ac00c860 con 0x7ff0bc06d300 2023-08-24T10:09:39.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.277+0000 7ff0a3fff640 1 -- 172.21.15.60:0/3398039189 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ff0ac00c9c0 con 0x7ff0bc06d300 2023-08-24T10:09:39.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.277+0000 7ff0a3fff640 1 -- 172.21.15.60:0/3398039189 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff0ac00cc60 con 0x7ff0bc06d300 2023-08-24T10:09:39.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.277+0000 7ff0c1aac640 1 -- 172.21.15.60:0/3398039189 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0bc128870 con 0x7ff0bc06d300 2023-08-24T10:09:39.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.277+0000 7ff0c1aac640 1 -- 172.21.15.60:0/3398039189 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0bc134b10 con 0x7ff0bc06d300 2023-08-24T10:09:39.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.278+0000 7ff0a3fff640 1 -- 172.21.15.60:0/3398039189 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7ff0ac011310 con 0x7ff0bc06d300 2023-08-24T10:09:39.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.278+0000 7ff0c1aac640 1 -- 172.21.15.60:0/3398039189 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0bc071a80 con 0x7ff0bc06d300 2023-08-24T10:09:39.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.278+0000 7ff0a3fff640 1 --2- 172.21.15.60:0/3398039189 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff0a8037130 0x7ff0a8039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:39.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.278+0000 7ff0a3fff640 1 -- 172.21.15.60:0/3398039189 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7ff0ac0469b0 con 0x7ff0bc06d300 2023-08-24T10:09:39.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.279+0000 7ff0baffd640 1 --2- 172.21.15.60:0/3398039189 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff0a8037130 0x7ff0a8039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:39.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.279+0000 7ff0baffd640 1 --2- 172.21.15.60:0/3398039189 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff0a8037130 0x7ff0a8039640 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:39.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.283+0000 7ff0a3fff640 1 -- 172.21.15.60:0/3398039189 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7ff0ac011e60 con 0x7ff0bc06d300 2023-08-24T10:09:39.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.455+0000 7ff0c1aac640 1 -- 172.21.15.60:0/3398039189 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff0bc098880 con 0x7ff0a8037130 2023-08-24T10:09:39.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.456+0000 7ff0a3fff640 1 -- 172.21.15.60:0/3398039189 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7ff0b0009d90 con 0x7ff0a8037130 2023-08-24T10:09:39.458 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:39.458 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:39.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.458+0000 7ff0c1aac640 1 -- 172.21.15.60:0/3398039189 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff0a8037130 msgr2=0x7ff0a8039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:39.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.458+0000 7ff0c1aac640 1 --2- 172.21.15.60:0/3398039189 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff0a8037130 0x7ff0a8039640 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:39.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.458+0000 7ff0c1aac640 1 -- 172.21.15.60:0/3398039189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0bc06d300 msgr2=0x7ff0bc1356b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:39.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.458+0000 7ff0c1aac640 1 --2- 172.21.15.60:0/3398039189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0bc06d300 0x7ff0bc1356b0 crc :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:39.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.458+0000 7ff0c1aac640 1 -- 172.21.15.60:0/3398039189 shutdown_connections 2023-08-24T10:09:39.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.458+0000 7ff0c1aac640 1 --2- 172.21.15.60:0/3398039189 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff0a8037130 0x7ff0a8039640 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:39.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.458+0000 7ff0c1aac640 1 --2- 172.21.15.60:0/3398039189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0bc06d300 0x7ff0bc1356b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:39.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.458+0000 7ff0c1aac640 1 -- 172.21.15.60:0/3398039189 >> 172.21.15.60:0/3398039189 conn(0x7ff0bc09e110 msgr2=0x7ff0bc129720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:39.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.459+0000 7ff0c1aac640 1 -- 172.21.15.60:0/3398039189 shutdown_connections 2023-08-24T10:09:39.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.459+0000 7ff0c1aac640 1 -- 172.21.15.60:0/3398039189 wait complete. 2023-08-24T10:09:39.471 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2023-08-24T10:09:37.548851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:39.473 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.580+0000 7f71f8f9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.584+0000 7f71f8f9e640 1 Processor -- start 2023-08-24T10:09:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.584+0000 7f71f8f9e640 1 -- start start 2023-08-24T10:09:39.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.585+0000 7f71f8f9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f71f4071e40 0x7f71f4096800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:39.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.585+0000 7f71f8f9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f40977f0 con 0x7f71f406f100 2023-08-24T10:09:39.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.585+0000 7f71f8f9e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f409a280 con 0x7f71f4071e40 2023-08-24T10:09:39.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.585+0000 7f71f259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f71f4071e40 0x7f71f4096800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:39.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.585+0000 7f71f259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f71f4071e40 0x7f71f4096800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37842/0 (socket says 172.21.15.60:37842) 2023-08-24T10:09:39.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.585+0000 7f71f259c640 1 -- 172.21.15.60:0/3135987966 learned_addr learned my addr 172.21.15.60:0/3135987966 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:39.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.585+0000 7f71f259c640 1 -- 172.21.15.60:0/3135987966 >> v1:172.21.15.60:6789/0 conn(0x7f71f406f100 legacy=0x7f71f406f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:39.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.585+0000 7f71f259c640 1 -- 172.21.15.60:0/3135987966 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f409a6b0 con 0x7f71f4071e40 2023-08-24T10:09:39.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.585+0000 7f71f259c640 1 --2- 172.21.15.60:0/3135987966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71f4071e40 0x7f71f4096800 crc :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ed9d8a8450c406 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:39.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.586+0000 7f71f1d9b640 1 -- 172.21.15.60:0/3135987966 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f71e4009dd0 con 0x7f71f4071e40 2023-08-24T10:09:39.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.586+0000 7f71f1d9b640 1 -- 172.21.15.60:0/3135987966 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f71e400e230 con 0x7f71f4071e40 2023-08-24T10:09:39.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.586+0000 7f71f1d9b640 1 -- 172.21.15.60:0/3135987966 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f71e400e4d0 con 0x7f71f4071e40 2023-08-24T10:09:39.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.586+0000 7f71f8f9e640 1 -- 172.21.15.60:0/3135987966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71f4071e40 msgr2=0x7f71f4096800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:39.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.586+0000 7f71f8f9e640 1 --2- 172.21.15.60:0/3135987966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71f4071e40 0x7f71f4096800 crc :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:39.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.586+0000 7f71f8f9e640 1 -- 172.21.15.60:0/3135987966 shutdown_connections 2023-08-24T10:09:39.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.586+0000 7f71f8f9e640 1 --2- 172.21.15.60:0/3135987966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71f4071e40 0x7f71f4096800 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:39.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.586+0000 7f71f8f9e640 1 -- 172.21.15.60:0/3135987966 >> 172.21.15.60:0/3135987966 conn(0x7f71f409e110 msgr2=0x7f71f409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:39.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.586+0000 7f71f8f9e640 1 -- 172.21.15.60:0/3135987966 shutdown_connections 2023-08-24T10:09:39.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.586+0000 7f71f8f9e640 1 -- 172.21.15.60:0/3135987966 wait complete. 2023-08-24T10:09:39.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.587+0000 7f71f8f9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:39.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.588+0000 7f71f8f9e640 1 Processor -- start 2023-08-24T10:09:39.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.588+0000 7f71f8f9e640 1 -- start start 2023-08-24T10:09:39.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.588+0000 7f71f8f9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71f406f100 0x7f71f406cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:39.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.588+0000 7f71f8f9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f40930d0 con 0x7f71f406f100 2023-08-24T10:09:39.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.588+0000 7f71f259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71f406f100 0x7f71f406cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:39.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.588+0000 7f71f259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71f406f100 0x7f71f406cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37854/0 (socket says 172.21.15.60:37854) 2023-08-24T10:09:39.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.588+0000 7f71f259c640 1 -- 172.21.15.60:0/4113601511 learned_addr learned my addr 172.21.15.60:0/4113601511 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:39.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.589+0000 7f71f259c640 1 -- 172.21.15.60:0/4113601511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f4125f70 con 0x7f71f406f100 2023-08-24T10:09:39.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.589+0000 7f71f259c640 1 --2- 172.21.15.60:0/4113601511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71f406f100 0x7f71f406cef0 crc :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:39.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.589+0000 7f71eaffd640 1 -- 172.21.15.60:0/4113601511 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f71e4014540 con 0x7f71f406f100 2023-08-24T10:09:39.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.589+0000 7f71eaffd640 1 -- 172.21.15.60:0/4113601511 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f71e40146a0 con 0x7f71f406f100 2023-08-24T10:09:39.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.589+0000 7f71f8f9e640 1 -- 172.21.15.60:0/4113601511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f4125340 con 0x7f71f406f100 2023-08-24T10:09:39.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.589+0000 7f71eaffd640 1 -- 172.21.15.60:0/4113601511 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f71e4014940 con 0x7f71f406f100 2023-08-24T10:09:39.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.589+0000 7f71f8f9e640 1 -- 172.21.15.60:0/4113601511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f4131040 con 0x7f71f406f100 2023-08-24T10:09:39.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.590+0000 7f71eaffd640 1 -- 172.21.15.60:0/4113601511 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f71e4014af0 con 0x7f71f406f100 2023-08-24T10:09:39.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.590+0000 7f71eaffd640 1 --2- 172.21.15.60:0/4113601511 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f71d4037130 0x7f71d4039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:39.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.590+0000 7f71eaffd640 1 -- 172.21.15.60:0/4113601511 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f71e4019040 con 0x7f71f406f100 2023-08-24T10:09:39.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.590+0000 7f71f8f9e640 1 -- 172.21.15.60:0/4113601511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f4125530 con 0x7f71f406f100 2023-08-24T10:09:39.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.590+0000 7f71ebfff640 1 --2- 172.21.15.60:0/4113601511 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f71d4037130 0x7f71d4039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:39.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.591+0000 7f71ebfff640 1 --2- 172.21.15.60:0/4113601511 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f71d4037130 0x7f71d4039640 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:39.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.595+0000 7f71eaffd640 1 -- 172.21.15.60:0/4113601511 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f71e4018190 con 0x7f71f406f100 2023-08-24T10:09:39.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.770+0000 7f71f8f9e640 1 -- 172.21.15.60:0/4113601511 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f71f4131290 con 0x7f71d4037130 2023-08-24T10:09:39.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.772+0000 7f71eaffd640 1 -- 172.21.15.60:0/4113601511 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f71f4131290 con 0x7f71d4037130 2023-08-24T10:09:39.774 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:39.775 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:39.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.774+0000 7f71f8f9e640 1 -- 172.21.15.60:0/4113601511 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f71d4037130 msgr2=0x7f71d4039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:39.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.774+0000 7f71f8f9e640 1 --2- 172.21.15.60:0/4113601511 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f71d4037130 0x7f71d4039640 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:39.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.774+0000 7f71f8f9e640 1 -- 172.21.15.60:0/4113601511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71f406f100 msgr2=0x7f71f406cef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:39.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.774+0000 7f71f8f9e640 1 --2- 172.21.15.60:0/4113601511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71f406f100 0x7f71f406cef0 crc :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:39.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.774+0000 7f71f8f9e640 1 -- 172.21.15.60:0/4113601511 shutdown_connections 2023-08-24T10:09:39.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.774+0000 7f71f8f9e640 1 --2- 172.21.15.60:0/4113601511 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f71d4037130 0x7f71d4039640 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:39.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.774+0000 7f71f8f9e640 1 --2- 172.21.15.60:0/4113601511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71f406f100 0x7f71f406cef0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:39.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.774+0000 7f71f8f9e640 1 -- 172.21.15.60:0/4113601511 >> 172.21.15.60:0/4113601511 conn(0x7f71f409e110 msgr2=0x7f71f40985b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:39.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.774+0000 7f71f8f9e640 1 -- 172.21.15.60:0/4113601511 shutdown_connections 2023-08-24T10:09:39.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.774+0000 7f71f8f9e640 1 -- 172.21.15.60:0/4113601511 wait complete. 2023-08-24T10:09:39.786 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2023-08-24T10:09:39.549294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:39.787 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:09:39.788 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:39.789 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:09:39.789 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:39.790 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:09:39.790 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:39.790 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:09:39.790 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:39.791 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:09:39.791 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:39.791 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:09:39.792 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:39.792 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:09:39.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:39.793 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:09:39.796 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:39.797 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:09:39.797 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:39.797 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:09:39.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.903+0000 7ff9c4cac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:39.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.906+0000 7ff9c4cac640 1 Processor -- start 2023-08-24T10:09:39.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.906+0000 7ff9c4cac640 1 -- start start 2023-08-24T10:09:39.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.907+0000 7ff9c4cac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff9c012ece0 0x7ff9c012f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:39.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.907+0000 7ff9c4cac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9c009a650 con 0x7ff9c012f630 2023-08-24T10:09:39.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.907+0000 7ff9c4cac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9c006da00 con 0x7ff9c012ece0 2023-08-24T10:09:39.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.907+0000 7ff9be59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff9c012ece0 0x7ff9c012f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:39.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.907+0000 7ff9be59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff9c012ece0 0x7ff9c012f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37856/0 (socket says 172.21.15.60:37856) 2023-08-24T10:09:39.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.907+0000 7ff9bdd9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff9c012f630 0x7ff9c0097fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41132/0 (socket says 172.21.15.60:41132) 2023-08-24T10:09:39.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.907+0000 7ff9be59c640 1 -- 172.21.15.60:0/3105082421 learned_addr learned my addr 172.21.15.60:0/3105082421 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:39.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.908+0000 7ff9be59c640 1 -- 172.21.15.60:0/3105082421 >> v1:172.21.15.60:6789/0 conn(0x7ff9c012f630 legacy=0x7ff9c0097fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:39.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.908+0000 7ff9be59c640 1 -- 172.21.15.60:0/3105082421 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c012ea10 con 0x7ff9c012ece0 2023-08-24T10:09:39.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.908+0000 7ff9be59c640 1 --2- 172.21.15.60:0/3105082421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c012ece0 0x7ff9c012f0f0 crc :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbf26657fea6c59 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:39.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.908+0000 7ff9bd59a640 1 -- 172.21.15.60:0/3105082421 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff9b800ad60 con 0x7ff9c012ece0 2023-08-24T10:09:39.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.908+0000 7ff9bd59a640 1 -- 172.21.15.60:0/3105082421 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ff9b800f340 con 0x7ff9c012ece0 2023-08-24T10:09:39.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.908+0000 7ff9bd59a640 1 -- 172.21.15.60:0/3105082421 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff9b800f5e0 con 0x7ff9c012ece0 2023-08-24T10:09:39.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.909+0000 7ff9c4cac640 1 -- 172.21.15.60:0/3105082421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c012ece0 msgr2=0x7ff9c012f0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:39.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.909+0000 7ff9c4cac640 1 --2- 172.21.15.60:0/3105082421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c012ece0 0x7ff9c012f0f0 crc :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:39.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.909+0000 7ff9c4cac640 1 -- 172.21.15.60:0/3105082421 shutdown_connections 2023-08-24T10:09:39.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.909+0000 7ff9c4cac640 1 --2- 172.21.15.60:0/3105082421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c012ece0 0x7ff9c012f0f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:39.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.909+0000 7ff9c4cac640 1 -- 172.21.15.60:0/3105082421 >> 172.21.15.60:0/3105082421 conn(0x7ff9c009e110 msgr2=0x7ff9c009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:39.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.909+0000 7ff9c4cac640 1 -- 172.21.15.60:0/3105082421 shutdown_connections 2023-08-24T10:09:39.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.909+0000 7ff9c4cac640 1 -- 172.21.15.60:0/3105082421 wait complete. 2023-08-24T10:09:39.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.909+0000 7ff9c4cac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:39.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.910+0000 7ff9c4cac640 1 Processor -- start 2023-08-24T10:09:39.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.910+0000 7ff9c4cac640 1 -- start start 2023-08-24T10:09:39.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.911+0000 7ff9c4cac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c012ece0 0x7ff9c0129410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:39.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.911+0000 7ff9c4cac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c01288f0 con 0x7ff9c012ece0 2023-08-24T10:09:39.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.911+0000 7ff9be59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c012ece0 0x7ff9c0129410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:39.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.911+0000 7ff9be59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c012ece0 0x7ff9c0129410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37866/0 (socket says 172.21.15.60:37866) 2023-08-24T10:09:39.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.911+0000 7ff9be59c640 1 -- 172.21.15.60:0/1101273792 learned_addr learned my addr 172.21.15.60:0/1101273792 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:39.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.911+0000 7ff9be59c640 1 -- 172.21.15.60:0/1101273792 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c0128ec0 con 0x7ff9c012ece0 2023-08-24T10:09:39.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.911+0000 7ff9be59c640 1 --2- 172.21.15.60:0/1101273792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c012ece0 0x7ff9c0129410 crc :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:39.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.911+0000 7ff9b6ffd640 1 -- 172.21.15.60:0/1101273792 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff9b8011420 con 0x7ff9c012ece0 2023-08-24T10:09:39.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.912+0000 7ff9b6ffd640 1 -- 172.21.15.60:0/1101273792 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ff9b8011580 con 0x7ff9c012ece0 2023-08-24T10:09:39.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.912+0000 7ff9c4cac640 1 -- 172.21.15.60:0/1101273792 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c0089610 con 0x7ff9c012ece0 2023-08-24T10:09:39.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.912+0000 7ff9b6ffd640 1 -- 172.21.15.60:0/1101273792 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff9b8011820 con 0x7ff9c012ece0 2023-08-24T10:09:39.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.912+0000 7ff9c4cac640 1 -- 172.21.15.60:0/1101273792 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c0128b20 con 0x7ff9c012ece0 2023-08-24T10:09:39.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.912+0000 7ff9b6ffd640 1 -- 172.21.15.60:0/1101273792 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7ff9b80119d0 con 0x7ff9c012ece0 2023-08-24T10:09:39.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.912+0000 7ff9c4cac640 1 -- 172.21.15.60:0/1101273792 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c0061800 con 0x7ff9c012ece0 2023-08-24T10:09:39.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.913+0000 7ff9b6ffd640 1 --2- 172.21.15.60:0/1101273792 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff9a0037130 0x7ff9a0039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:39.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.913+0000 7ff9b6ffd640 1 -- 172.21.15.60:0/1101273792 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7ff9b8017040 con 0x7ff9c012ece0 2023-08-24T10:09:39.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.913+0000 7ff9bdd9b640 1 --2- 172.21.15.60:0/1101273792 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff9a0037130 0x7ff9a0039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:39.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.913+0000 7ff9bdd9b640 1 --2- 172.21.15.60:0/1101273792 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff9a0037130 0x7ff9a0039640 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:39.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:39.917+0000 7ff9b6ffd640 1 -- 172.21.15.60:0/1101273792 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7ff9b800b5d0 con 0x7ff9c012ece0 2023-08-24T10:09:40.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.163+0000 7ff9c4cac640 1 -- 172.21.15.60:0/1101273792 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff9c0129d70 con 0x7ff9c012ece0 2023-08-24T10:09:40.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.163+0000 7ff9b6ffd640 1 -- 172.21.15.60:0/1101273792 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7ff9b800b5d0 con 0x7ff9c012ece0 2023-08-24T10:09:40.165 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:40.165 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":290,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:09:40.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.165+0000 7ff9c4cac640 1 -- 172.21.15.60:0/1101273792 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff9a0037130 msgr2=0x7ff9a0039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:40.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.165+0000 7ff9c4cac640 1 --2- 172.21.15.60:0/1101273792 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff9a0037130 0x7ff9a0039640 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:40.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.165+0000 7ff9c4cac640 1 -- 172.21.15.60:0/1101273792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c012ece0 msgr2=0x7ff9c0129410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:40.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.165+0000 7ff9c4cac640 1 --2- 172.21.15.60:0/1101273792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c012ece0 0x7ff9c0129410 crc :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:40.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.165+0000 7ff9c4cac640 1 -- 172.21.15.60:0/1101273792 shutdown_connections 2023-08-24T10:09:40.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.165+0000 7ff9c4cac640 1 --2- 172.21.15.60:0/1101273792 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff9a0037130 0x7ff9a0039640 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:40.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.165+0000 7ff9c4cac640 1 --2- 172.21.15.60:0/1101273792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9c012ece0 0x7ff9c0129410 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:40.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.165+0000 7ff9c4cac640 1 -- 172.21.15.60:0/1101273792 >> 172.21.15.60:0/1101273792 conn(0x7ff9c009e110 msgr2=0x7ff9c0097710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:40.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.165+0000 7ff9c4cac640 1 -- 172.21.15.60:0/1101273792 shutdown_connections 2023-08-24T10:09:40.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.165+0000 7ff9c4cac640 1 -- 172.21.15.60:0/1101273792 wait complete. 2023-08-24T10:09:40.177 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:09:40.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:40.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.283+0000 7f423011e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:40.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.287+0000 7f423011e640 1 Processor -- start 2023-08-24T10:09:40.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.287+0000 7f423011e640 1 -- start start 2023-08-24T10:09:40.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.288+0000 7f423011e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4228132630 0x7f4228134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:40.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.288+0000 7f423011e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422806e320 con 0x7f4228096860 2023-08-24T10:09:40.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.288+0000 7f423011e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f422806db70 con 0x7f4228132630 2023-08-24T10:09:40.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.288+0000 7f422d6b9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4228132630 0x7f4228134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:40.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.288+0000 7f422d6b9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4228132630 0x7f4228134a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37882/0 (socket says 172.21.15.60:37882) 2023-08-24T10:09:40.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.288+0000 7f422deba640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4228096860 0x7f4228096c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41142/0 (socket says 172.21.15.60:41142) 2023-08-24T10:09:40.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.288+0000 7f422d6b9640 1 -- 172.21.15.60:0/2645649538 learned_addr learned my addr 172.21.15.60:0/2645649538 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:40.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.288+0000 7f422d6b9640 1 -- 172.21.15.60:0/2645649538 >> v1:172.21.15.60:6789/0 conn(0x7f4228096860 legacy=0x7f4228096c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:40.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.288+0000 7f422d6b9640 1 -- 172.21.15.60:0/2645649538 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422806cef0 con 0x7f4228132630 2023-08-24T10:09:40.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.289+0000 7f422d6b9640 1 --2- 172.21.15.60:0/2645649538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4228132630 0x7f4228134a50 crc :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62fd7df154ba6872 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:40.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.289+0000 7f422ceb8640 1 -- 172.21.15.60:0/2645649538 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f421c009dd0 con 0x7f4228132630 2023-08-24T10:09:40.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.289+0000 7f422ceb8640 1 -- 172.21.15.60:0/2645649538 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f421c00e230 con 0x7f4228132630 2023-08-24T10:09:40.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.289+0000 7f422ceb8640 1 -- 172.21.15.60:0/2645649538 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f421c00e4d0 con 0x7f4228132630 2023-08-24T10:09:40.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.289+0000 7f423011e640 1 -- 172.21.15.60:0/2645649538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4228132630 msgr2=0x7f4228134a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:40.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.289+0000 7f423011e640 1 --2- 172.21.15.60:0/2645649538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4228132630 0x7f4228134a50 crc :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:40.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.289+0000 7f423011e640 1 -- 172.21.15.60:0/2645649538 shutdown_connections 2023-08-24T10:09:40.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.289+0000 7f423011e640 1 --2- 172.21.15.60:0/2645649538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4228132630 0x7f4228134a50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:40.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.289+0000 7f423011e640 1 -- 172.21.15.60:0/2645649538 >> 172.21.15.60:0/2645649538 conn(0x7f422809e110 msgr2=0x7f422809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:40.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.290+0000 7f423011e640 1 -- 172.21.15.60:0/2645649538 shutdown_connections 2023-08-24T10:09:40.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.290+0000 7f423011e640 1 -- 172.21.15.60:0/2645649538 wait complete. 2023-08-24T10:09:40.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.290+0000 7f423011e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:40.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.291+0000 7f423011e640 1 Processor -- start 2023-08-24T10:09:40.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.291+0000 7f423011e640 1 -- start start 2023-08-24T10:09:40.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.291+0000 7f423011e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4228096860 0x7f42281c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:40.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.291+0000 7f423011e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4228098ce0 con 0x7f4228096860 2023-08-24T10:09:40.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.291+0000 7f422deba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4228096860 0x7f42281c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:40.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.291+0000 7f422deba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4228096860 0x7f42281c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37888/0 (socket says 172.21.15.60:37888) 2023-08-24T10:09:40.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.291+0000 7f422deba640 1 -- 172.21.15.60:0/518726772 learned_addr learned my addr 172.21.15.60:0/518726772 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:40.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.292+0000 7f422deba640 1 -- 172.21.15.60:0/518726772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422812b2a0 con 0x7f4228096860 2023-08-24T10:09:40.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.292+0000 7f422deba640 1 --2- 172.21.15.60:0/518726772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4228096860 0x7f42281c52a0 crc :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:40.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.292+0000 7f42267fc640 1 -- 172.21.15.60:0/518726772 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f42200070b0 con 0x7f4228096860 2023-08-24T10:09:40.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.292+0000 7f42267fc640 1 -- 172.21.15.60:0/518726772 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4220007210 con 0x7f4228096860 2023-08-24T10:09:40.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.292+0000 7f42267fc640 1 -- 172.21.15.60:0/518726772 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f42200074b0 con 0x7f4228096860 2023-08-24T10:09:40.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.292+0000 7f423011e640 1 -- 172.21.15.60:0/518726772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422812b9e0 con 0x7f4228096860 2023-08-24T10:09:40.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.292+0000 7f423011e640 1 -- 172.21.15.60:0/518726772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42281c4e70 con 0x7f4228096860 2023-08-24T10:09:40.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.293+0000 7f42267fc640 1 -- 172.21.15.60:0/518726772 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f42200057e0 con 0x7f4228096860 2023-08-24T10:09:40.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.293+0000 7f423011e640 1 -- 172.21.15.60:0/518726772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4228061800 con 0x7f4228096860 2023-08-24T10:09:40.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.293+0000 7f42267fc640 1 --2- 172.21.15.60:0/518726772 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4214037130 0x7f4214039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:40.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.293+0000 7f42267fc640 1 -- 172.21.15.60:0/518726772 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f42200462f0 con 0x7f4228096860 2023-08-24T10:09:40.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.293+0000 7f422d6b9640 1 --2- 172.21.15.60:0/518726772 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4214037130 0x7f4214039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:40.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.294+0000 7f422d6b9640 1 --2- 172.21.15.60:0/518726772 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4214037130 0x7f4214039640 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:40.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.298+0000 7f42267fc640 1 -- 172.21.15.60:0/518726772 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f422000d6f0 con 0x7f4228096860 2023-08-24T10:09:40.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.472+0000 7f423011e640 1 -- 172.21.15.60:0/518726772 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f422806d190 con 0x7f4214037130 2023-08-24T10:09:40.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.473+0000 7f42267fc640 1 -- 172.21.15.60:0/518726772 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f421c009e40 con 0x7f4214037130 2023-08-24T10:09:40.474 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:40.475 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:40.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.475+0000 7f423011e640 1 -- 172.21.15.60:0/518726772 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4214037130 msgr2=0x7f4214039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:40.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.475+0000 7f423011e640 1 --2- 172.21.15.60:0/518726772 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4214037130 0x7f4214039640 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:40.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.475+0000 7f423011e640 1 -- 172.21.15.60:0/518726772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4228096860 msgr2=0x7f42281c52a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:40.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.475+0000 7f423011e640 1 --2- 172.21.15.60:0/518726772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4228096860 0x7f42281c52a0 crc :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:40.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.475+0000 7f423011e640 1 -- 172.21.15.60:0/518726772 shutdown_connections 2023-08-24T10:09:40.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.475+0000 7f423011e640 1 --2- 172.21.15.60:0/518726772 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f4214037130 0x7f4214039640 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:40.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.475+0000 7f423011e640 1 --2- 172.21.15.60:0/518726772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4228096860 0x7f42281c52a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:40.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.475+0000 7f423011e640 1 -- 172.21.15.60:0/518726772 >> 172.21.15.60:0/518726772 conn(0x7f422809e110 msgr2=0x7f42280950c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:40.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.475+0000 7f423011e640 1 -- 172.21.15.60:0/518726772 shutdown_connections 2023-08-24T10:09:40.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:40.475+0000 7f423011e640 1 -- 172.21.15.60:0/518726772 wait complete. 2023-08-24T10:09:40.486 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2023-08-24T10:09:39.549294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:43.255 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~276s 2023-08-24T10:09:43.255 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~276s 2023-08-24T10:09:43.256 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~276s 2023-08-24T10:09:43.256 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~276s 2023-08-24T10:09:43.488 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:43.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.608+0000 7f1fa8c30640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:43.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.612+0000 7f1fa8c30640 1 Processor -- start 2023-08-24T10:09:43.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.612+0000 7f1fa8c30640 1 -- start start 2023-08-24T10:09:43.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.612+0000 7f1fa8c30640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1fa4071e40 0x7f1fa4096800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:43.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.612+0000 7f1fa8c30640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa40977f0 con 0x7f1fa406f100 2023-08-24T10:09:43.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.612+0000 7f1fa8c30640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa409a280 con 0x7f1fa4071e40 2023-08-24T10:09:43.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.612+0000 7f1fa259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1fa4071e40 0x7f1fa4096800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:43.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.613+0000 7f1fa259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1fa4071e40 0x7f1fa4096800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37894/0 (socket says 172.21.15.60:37894) 2023-08-24T10:09:43.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.613+0000 7f1fa259c640 1 -- 172.21.15.60:0/826309319 learned_addr learned my addr 172.21.15.60:0/826309319 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:43.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.613+0000 7f1fa259c640 1 -- 172.21.15.60:0/826309319 >> v1:172.21.15.60:6789/0 conn(0x7f1fa406f100 legacy=0x7f1fa406f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:43.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.613+0000 7f1fa259c640 1 -- 172.21.15.60:0/826309319 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa409a6b0 con 0x7f1fa4071e40 2023-08-24T10:09:43.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.613+0000 7f1fa259c640 1 --2- 172.21.15.60:0/826309319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa4071e40 0x7f1fa4096800 crc :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7323bef893d591 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:43.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.613+0000 7f1fa1d9b640 1 -- 172.21.15.60:0/826309319 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1f94009dd0 con 0x7f1fa4071e40 2023-08-24T10:09:43.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.613+0000 7f1fa1d9b640 1 -- 172.21.15.60:0/826309319 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1f9400e230 con 0x7f1fa4071e40 2023-08-24T10:09:43.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.613+0000 7f1fa1d9b640 1 -- 172.21.15.60:0/826309319 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1f9400e4d0 con 0x7f1fa4071e40 2023-08-24T10:09:43.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.614+0000 7f1fa8c30640 1 -- 172.21.15.60:0/826309319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa4071e40 msgr2=0x7f1fa4096800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:43.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.614+0000 7f1fa8c30640 1 --2- 172.21.15.60:0/826309319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa4071e40 0x7f1fa4096800 crc :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:43.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.614+0000 7f1fa8c30640 1 -- 172.21.15.60:0/826309319 shutdown_connections 2023-08-24T10:09:43.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.614+0000 7f1fa8c30640 1 --2- 172.21.15.60:0/826309319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa4071e40 0x7f1fa4096800 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:43.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.614+0000 7f1fa8c30640 1 -- 172.21.15.60:0/826309319 >> 172.21.15.60:0/826309319 conn(0x7f1fa409e110 msgr2=0x7f1fa409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:43.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.614+0000 7f1fa8c30640 1 -- 172.21.15.60:0/826309319 shutdown_connections 2023-08-24T10:09:43.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.614+0000 7f1fa8c30640 1 -- 172.21.15.60:0/826309319 wait complete. 2023-08-24T10:09:43.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.614+0000 7f1fa8c30640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:43.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.615+0000 7f1fa8c30640 1 Processor -- start 2023-08-24T10:09:43.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.615+0000 7f1fa8c30640 1 -- start start 2023-08-24T10:09:43.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.616+0000 7f1fa8c30640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa406f100 0x7f1fa41c19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:43.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.616+0000 7f1fa8c30640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa408ffe0 con 0x7f1fa406f100 2023-08-24T10:09:43.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.616+0000 7f1fa259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa406f100 0x7f1fa41c19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:43.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.616+0000 7f1fa259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa406f100 0x7f1fa41c19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37910/0 (socket says 172.21.15.60:37910) 2023-08-24T10:09:43.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.616+0000 7f1fa259c640 1 -- 172.21.15.60:0/1989678475 learned_addr learned my addr 172.21.15.60:0/1989678475 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:43.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.617+0000 7f1fa259c640 1 -- 172.21.15.60:0/1989678475 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa41c14b0 con 0x7f1fa406f100 2023-08-24T10:09:43.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.617+0000 7f1fa259c640 1 --2- 172.21.15.60:0/1989678475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa406f100 0x7f1fa41c19f0 crc :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:43.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.617+0000 7f1f9b7fe640 1 -- 172.21.15.60:0/1989678475 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1f94014540 con 0x7f1fa406f100 2023-08-24T10:09:43.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.617+0000 7f1f9b7fe640 1 -- 172.21.15.60:0/1989678475 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1f940146a0 con 0x7f1fa406f100 2023-08-24T10:09:43.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.617+0000 7f1fa8c30640 1 -- 172.21.15.60:0/1989678475 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa4090720 con 0x7f1fa406f100 2023-08-24T10:09:43.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.617+0000 7f1f9b7fe640 1 -- 172.21.15.60:0/1989678475 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1f94014940 con 0x7f1fa406f100 2023-08-24T10:09:43.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.617+0000 7f1fa8c30640 1 -- 172.21.15.60:0/1989678475 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa4092c10 con 0x7f1fa406f100 2023-08-24T10:09:43.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.618+0000 7f1f9b7fe640 1 -- 172.21.15.60:0/1989678475 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f1f94014af0 con 0x7f1fa406f100 2023-08-24T10:09:43.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.618+0000 7f1fa8c30640 1 -- 172.21.15.60:0/1989678475 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa4061800 con 0x7f1fa406f100 2023-08-24T10:09:43.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.618+0000 7f1f9b7fe640 1 --2- 172.21.15.60:0/1989678475 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1f84037180 0x7f1f84039690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:43.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.618+0000 7f1f9b7fe640 1 -- 172.21.15.60:0/1989678475 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f1f94019040 con 0x7f1fa406f100 2023-08-24T10:09:43.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.619+0000 7f1f99d9b640 1 --2- 172.21.15.60:0/1989678475 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1f84037180 0x7f1f84039690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:43.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.619+0000 7f1f99d9b640 1 --2- 172.21.15.60:0/1989678475 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1f84037180 0x7f1f84039690 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:43.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.623+0000 7f1f9b7fe640 1 -- 172.21.15.60:0/1989678475 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f1f94018190 con 0x7f1fa406f100 2023-08-24T10:09:43.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.795+0000 7f1fa8c30640 1 -- 172.21.15.60:0/1989678475 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1fa4097a80 con 0x7f1f84037180 2023-08-24T10:09:43.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.797+0000 7f1f9b7fe640 1 -- 172.21.15.60:0/1989678475 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f1f8c001650 con 0x7f1f84037180 2023-08-24T10:09:43.797 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:43.798 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:43.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.798+0000 7f1fa8c30640 1 -- 172.21.15.60:0/1989678475 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1f84037180 msgr2=0x7f1f84039690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:43.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.798+0000 7f1fa8c30640 1 --2- 172.21.15.60:0/1989678475 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1f84037180 0x7f1f84039690 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:43.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.798+0000 7f1fa8c30640 1 -- 172.21.15.60:0/1989678475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa406f100 msgr2=0x7f1fa41c19f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:43.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.798+0000 7f1fa8c30640 1 --2- 172.21.15.60:0/1989678475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa406f100 0x7f1fa41c19f0 crc :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:43.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.798+0000 7f1fa8c30640 1 -- 172.21.15.60:0/1989678475 shutdown_connections 2023-08-24T10:09:43.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.798+0000 7f1fa8c30640 1 --2- 172.21.15.60:0/1989678475 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1f84037180 0x7f1f84039690 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:43.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.798+0000 7f1fa8c30640 1 --2- 172.21.15.60:0/1989678475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa406f100 0x7f1fa41c19f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:43.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.798+0000 7f1fa8c30640 1 -- 172.21.15.60:0/1989678475 >> 172.21.15.60:0/1989678475 conn(0x7f1fa409e110 msgr2=0x7f1fa40985b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:43.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.798+0000 7f1fa8c30640 1 -- 172.21.15.60:0/1989678475 shutdown_connections 2023-08-24T10:09:43.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.798+0000 7f1fa8c30640 1 -- 172.21.15.60:0/1989678475 wait complete. 2023-08-24T10:09:43.810 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2023-08-24T10:09:43.550307+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:43.811 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:43.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.917+0000 7fe36d4ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:43.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.921+0000 7fe36d4ec640 1 Processor -- start 2023-08-24T10:09:43.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.921+0000 7fe36d4ec640 1 -- start start 2023-08-24T10:09:43.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.922+0000 7fe36d4ec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe368132b50 0x7fe368132f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:43.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.922+0000 7fe36d4ec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe368132610 con 0x7fe36812f110 2023-08-24T10:09:43.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.922+0000 7fe36d4ec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36809c260 con 0x7fe368132b50 2023-08-24T10:09:43.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.922+0000 7fe366ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe368132b50 0x7fe368132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:43.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.922+0000 7fe366ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe368132b50 0x7fe368132f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37926/0 (socket says 172.21.15.60:37926) 2023-08-24T10:09:43.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.922+0000 7fe366ffd640 1 -- 172.21.15.60:0/3867961779 learned_addr learned my addr 172.21.15.60:0/3867961779 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:43.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.923+0000 7fe366ffd640 1 -- 172.21.15.60:0/3867961779 >> v1:172.21.15.60:6789/0 conn(0x7fe36812f110 legacy=0x7fe36812f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:43.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.923+0000 7fe366ffd640 1 -- 172.21.15.60:0/3867961779 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe368132840 con 0x7fe368132b50 2023-08-24T10:09:43.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.923+0000 7fe366ffd640 1 --2- 172.21.15.60:0/3867961779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe368132b50 0x7fe368132f60 crc :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9b5651b814b0bd server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:43.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.923+0000 7fe365ffb640 1 -- 172.21.15.60:0/3867961779 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe35800e040 con 0x7fe368132b50 2023-08-24T10:09:43.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.923+0000 7fe365ffb640 1 -- 172.21.15.60:0/3867961779 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fe3580091e0 con 0x7fe368132b50 2023-08-24T10:09:43.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.923+0000 7fe365ffb640 1 -- 172.21.15.60:0/3867961779 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe358007730 con 0x7fe368132b50 2023-08-24T10:09:43.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.924+0000 7fe36d4ec640 1 -- 172.21.15.60:0/3867961779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe368132b50 msgr2=0x7fe368132f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:43.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.924+0000 7fe36d4ec640 1 --2- 172.21.15.60:0/3867961779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe368132b50 0x7fe368132f60 crc :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:43.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.924+0000 7fe36d4ec640 1 -- 172.21.15.60:0/3867961779 shutdown_connections 2023-08-24T10:09:43.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.924+0000 7fe36d4ec640 1 --2- 172.21.15.60:0/3867961779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe368132b50 0x7fe368132f60 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:43.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.924+0000 7fe36d4ec640 1 -- 172.21.15.60:0/3867961779 >> 172.21.15.60:0/3867961779 conn(0x7fe3680a26a0 msgr2=0x7fe3680a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:43.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.924+0000 7fe36d4ec640 1 -- 172.21.15.60:0/3867961779 shutdown_connections 2023-08-24T10:09:43.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.924+0000 7fe36d4ec640 1 -- 172.21.15.60:0/3867961779 wait complete. 2023-08-24T10:09:43.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.924+0000 7fe36d4ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:43.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.925+0000 7fe36d4ec640 1 Processor -- start 2023-08-24T10:09:43.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.925+0000 7fe36d4ec640 1 -- start start 2023-08-24T10:09:43.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.926+0000 7fe36d4ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe36812f110 0x7fe36809fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:43.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.926+0000 7fe36d4ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36812cc80 con 0x7fe36812f110 2023-08-24T10:09:43.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.926+0000 7fe366ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe36812f110 0x7fe36809fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:43.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.926+0000 7fe366ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe36812f110 0x7fe36809fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37936/0 (socket says 172.21.15.60:37936) 2023-08-24T10:09:43.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.926+0000 7fe366ffd640 1 -- 172.21.15.60:0/4278974858 learned_addr learned my addr 172.21.15.60:0/4278974858 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:43.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.926+0000 7fe366ffd640 1 -- 172.21.15.60:0/4278974858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36809f400 con 0x7fe36812f110 2023-08-24T10:09:43.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.926+0000 7fe366ffd640 1 --2- 172.21.15.60:0/4278974858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe36812f110 0x7fe36809fcd0 crc :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:43.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.926+0000 7fe35f7fe640 1 -- 172.21.15.60:0/4278974858 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe35800e040 con 0x7fe36812f110 2023-08-24T10:09:43.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.926+0000 7fe35f7fe640 1 -- 172.21.15.60:0/4278974858 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fe358010800 con 0x7fe36812f110 2023-08-24T10:09:43.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.926+0000 7fe36d4ec640 1 -- 172.21.15.60:0/4278974858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3680a18e0 con 0x7fe36812f110 2023-08-24T10:09:43.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.926+0000 7fe35f7fe640 1 -- 172.21.15.60:0/4278974858 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe358010aa0 con 0x7fe36812f110 2023-08-24T10:09:43.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.926+0000 7fe36d4ec640 1 -- 172.21.15.60:0/4278974858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36806a1d0 con 0x7fe36812f110 2023-08-24T10:09:43.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.927+0000 7fe35f7fe640 1 -- 172.21.15.60:0/4278974858 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fe358010c00 con 0x7fe36812f110 2023-08-24T10:09:43.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.927+0000 7fe35f7fe640 1 --2- 172.21.15.60:0/4278974858 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe348037130 0x7fe348039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:43.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.927+0000 7fe35f7fe640 1 -- 172.21.15.60:0/4278974858 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fe3580479e0 con 0x7fe36812f110 2023-08-24T10:09:43.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.927+0000 7fe36d4ec640 1 -- 172.21.15.60:0/4278974858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36811c2b0 con 0x7fe36812f110 2023-08-24T10:09:43.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.928+0000 7fe3667fc640 1 --2- 172.21.15.60:0/4278974858 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe348037130 0x7fe348039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:43.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.928+0000 7fe3667fc640 1 --2- 172.21.15.60:0/4278974858 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe348037130 0x7fe348039640 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:43.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:43.932+0000 7fe35f7fe640 1 -- 172.21.15.60:0/4278974858 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fe35800e1a0 con 0x7fe36812f110 2023-08-24T10:09:44.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.106+0000 7fe36d4ec640 1 -- 172.21.15.60:0/4278974858 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3681334a0 con 0x7fe348037130 2023-08-24T10:09:44.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.107+0000 7fe35f7fe640 1 -- 172.21.15.60:0/4278974858 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7fe350001650 con 0x7fe348037130 2023-08-24T10:09:44.109 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:44.110 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:44.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.109+0000 7fe36d4ec640 1 -- 172.21.15.60:0/4278974858 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe348037130 msgr2=0x7fe348039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:44.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.109+0000 7fe36d4ec640 1 --2- 172.21.15.60:0/4278974858 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe348037130 0x7fe348039640 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:44.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.109+0000 7fe36d4ec640 1 -- 172.21.15.60:0/4278974858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe36812f110 msgr2=0x7fe36809fcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:44.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.109+0000 7fe36d4ec640 1 --2- 172.21.15.60:0/4278974858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe36812f110 0x7fe36809fcd0 crc :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:44.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.109+0000 7fe36d4ec640 1 -- 172.21.15.60:0/4278974858 shutdown_connections 2023-08-24T10:09:44.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.109+0000 7fe36d4ec640 1 --2- 172.21.15.60:0/4278974858 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fe348037130 0x7fe348039640 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:44.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.109+0000 7fe36d4ec640 1 --2- 172.21.15.60:0/4278974858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe36812f110 0x7fe36809fcd0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:44.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.109+0000 7fe36d4ec640 1 -- 172.21.15.60:0/4278974858 >> 172.21.15.60:0/4278974858 conn(0x7fe3680a26a0 msgr2=0x7fe3681ba920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:44.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.109+0000 7fe36d4ec640 1 -- 172.21.15.60:0/4278974858 shutdown_connections 2023-08-24T10:09:44.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.109+0000 7fe36d4ec640 1 -- 172.21.15.60:0/4278974858 wait complete. 2023-08-24T10:09:44.121 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2023-08-24T10:09:43.550307+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:44.122 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:09:44.123 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:44.123 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:09:44.123 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:44.124 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:09:44.124 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:44.124 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:09:44.124 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:44.125 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:09:44.125 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:44.125 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:09:44.126 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:44.126 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:09:44.126 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:44.126 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:09:44.127 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:44.127 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:09:44.127 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:44.127 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:09:44.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.235+0000 7f542819e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:44.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.237+0000 7f542819e640 1 Processor -- start 2023-08-24T10:09:44.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.238+0000 7f542819e640 1 -- start start 2023-08-24T10:09:44.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.238+0000 7f542819e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5420096410 0x7f5420096820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:44.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.238+0000 7f542819e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542009a1e0 con 0x7f542006f170 2023-08-24T10:09:44.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.238+0000 7f542819e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5420097af0 con 0x7f5420096410 2023-08-24T10:09:44.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.239+0000 7f5425f3a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5420096410 0x7f5420096820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:44.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.239+0000 7f5425f3a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5420096410 0x7f5420096820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37948/0 (socket says 172.21.15.60:37948) 2023-08-24T10:09:44.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.239+0000 7f5425f3a640 1 -- 172.21.15.60:0/1982502064 learned_addr learned my addr 172.21.15.60:0/1982502064 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:44.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.239+0000 7f5425739640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f542006f170 0x7f542006f580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41182/0 (socket says 172.21.15.60:41182) 2023-08-24T10:09:44.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.239+0000 7f5425f3a640 1 -- 172.21.15.60:0/1982502064 >> v1:172.21.15.60:6789/0 conn(0x7f542006f170 legacy=0x7f542006f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:44.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.239+0000 7f5425f3a640 1 -- 172.21.15.60:0/1982502064 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542009a410 con 0x7f5420096410 2023-08-24T10:09:44.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.239+0000 7f5425f3a640 1 --2- 172.21.15.60:0/1982502064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5420096410 0x7f5420096820 crc :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e423b268391dc7 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:44.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.239+0000 7f5424f38640 1 -- 172.21.15.60:0/1982502064 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f541c009dd0 con 0x7f5420096410 2023-08-24T10:09:44.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.239+0000 7f5424f38640 1 -- 172.21.15.60:0/1982502064 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f541c00e230 con 0x7f5420096410 2023-08-24T10:09:44.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.239+0000 7f5424f38640 1 -- 172.21.15.60:0/1982502064 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f541c00e4d0 con 0x7f5420096410 2023-08-24T10:09:44.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.240+0000 7f542819e640 1 -- 172.21.15.60:0/1982502064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5420096410 msgr2=0x7f5420096820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:44.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.240+0000 7f542819e640 1 --2- 172.21.15.60:0/1982502064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5420096410 0x7f5420096820 crc :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:44.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.240+0000 7f542819e640 1 -- 172.21.15.60:0/1982502064 shutdown_connections 2023-08-24T10:09:44.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.240+0000 7f542819e640 1 --2- 172.21.15.60:0/1982502064 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5420096410 0x7f5420096820 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:44.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.240+0000 7f542819e640 1 -- 172.21.15.60:0/1982502064 >> 172.21.15.60:0/1982502064 conn(0x7f542009e110 msgr2=0x7f542009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:44.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.240+0000 7f542819e640 1 -- 172.21.15.60:0/1982502064 shutdown_connections 2023-08-24T10:09:44.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.240+0000 7f542819e640 1 -- 172.21.15.60:0/1982502064 wait complete. 2023-08-24T10:09:44.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.240+0000 7f542819e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:44.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.241+0000 7f542819e640 1 Processor -- start 2023-08-24T10:09:44.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.241+0000 7f542819e640 1 -- start start 2023-08-24T10:09:44.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.242+0000 7f542819e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f542006f170 0x7f542012a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:44.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.242+0000 7f542819e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542012bbc0 con 0x7f542006f170 2023-08-24T10:09:44.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.242+0000 7f5425f3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f542006f170 0x7f542012a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:44.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.242+0000 7f5425f3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f542006f170 0x7f542012a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37962/0 (socket says 172.21.15.60:37962) 2023-08-24T10:09:44.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.242+0000 7f5425f3a640 1 -- 172.21.15.60:0/893340966 learned_addr learned my addr 172.21.15.60:0/893340966 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:44.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.242+0000 7f5425f3a640 1 -- 172.21.15.60:0/893340966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542012c3d0 con 0x7f542006f170 2023-08-24T10:09:44.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.242+0000 7f5425f3a640 1 --2- 172.21.15.60:0/893340966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f542006f170 0x7f542012a650 crc :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:44.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.242+0000 7f54167fc640 1 -- 172.21.15.60:0/893340966 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f541c014410 con 0x7f542006f170 2023-08-24T10:09:44.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.242+0000 7f54167fc640 1 -- 172.21.15.60:0/893340966 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f541c014570 con 0x7f542006f170 2023-08-24T10:09:44.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.242+0000 7f54167fc640 1 -- 172.21.15.60:0/893340966 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f541c014810 con 0x7f542006f170 2023-08-24T10:09:44.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.242+0000 7f542819e640 1 -- 172.21.15.60:0/893340966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542012a290 con 0x7f542006f170 2023-08-24T10:09:44.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.242+0000 7f542819e640 1 -- 172.21.15.60:0/893340966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542012dc50 con 0x7f542006f170 2023-08-24T10:09:44.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.243+0000 7f54167fc640 1 -- 172.21.15.60:0/893340966 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f541c0149c0 con 0x7f542006f170 2023-08-24T10:09:44.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.243+0000 7f542819e640 1 -- 172.21.15.60:0/893340966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542012b760 con 0x7f542006f170 2023-08-24T10:09:44.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.243+0000 7f54167fc640 1 --2- 172.21.15.60:0/893340966 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5404037130 0x7f5404039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:44.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.243+0000 7f54167fc640 1 -- 172.21.15.60:0/893340966 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f541c019040 con 0x7f542006f170 2023-08-24T10:09:44.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.244+0000 7f5425739640 1 --2- 172.21.15.60:0/893340966 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5404037130 0x7f5404039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:44.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.244+0000 7f5425739640 1 --2- 172.21.15.60:0/893340966 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5404037130 0x7f5404039640 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:44.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.248+0000 7f54167fc640 1 -- 172.21.15.60:0/893340966 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f541c00f8f0 con 0x7f542006f170 2023-08-24T10:09:44.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.491+0000 7f542819e640 1 -- 172.21.15.60:0/893340966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f5420061800 con 0x7f542006f170 2023-08-24T10:09:44.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.491+0000 7f54167fc640 1 -- 172.21.15.60:0/893340966 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f541c00f710 con 0x7f542006f170 2023-08-24T10:09:44.493 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:44.494 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":295,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:09:44.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.493+0000 7f542819e640 1 -- 172.21.15.60:0/893340966 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5404037130 msgr2=0x7f5404039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:44.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.493+0000 7f542819e640 1 --2- 172.21.15.60:0/893340966 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5404037130 0x7f5404039640 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:44.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.493+0000 7f542819e640 1 -- 172.21.15.60:0/893340966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f542006f170 msgr2=0x7f542012a650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:44.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.493+0000 7f542819e640 1 --2- 172.21.15.60:0/893340966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f542006f170 0x7f542012a650 crc :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:44.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.493+0000 7f542819e640 1 -- 172.21.15.60:0/893340966 shutdown_connections 2023-08-24T10:09:44.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.493+0000 7f542819e640 1 --2- 172.21.15.60:0/893340966 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5404037130 0x7f5404039640 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:44.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.493+0000 7f542819e640 1 --2- 172.21.15.60:0/893340966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f542006f170 0x7f542012a650 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:44.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.493+0000 7f542819e640 1 -- 172.21.15.60:0/893340966 >> 172.21.15.60:0/893340966 conn(0x7f542009e110 msgr2=0x7f542009b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:44.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.493+0000 7f542819e640 1 -- 172.21.15.60:0/893340966 shutdown_connections 2023-08-24T10:09:44.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.494+0000 7f542819e640 1 -- 172.21.15.60:0/893340966 wait complete. 2023-08-24T10:09:44.505 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:09:44.506 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:44.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.613+0000 7f43d331e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:44.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.616+0000 7f43d331e640 1 Processor -- start 2023-08-24T10:09:44.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.617+0000 7f43d331e640 1 -- start start 2023-08-24T10:09:44.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.617+0000 7f43d331e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43cc070110 0x7f43cc1356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:44.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.617+0000 7f43d331e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43cc096f10 con 0x7f43cc071e40 2023-08-24T10:09:44.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.617+0000 7f43d331e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43cc0977f0 con 0x7f43cc070110 2023-08-24T10:09:44.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.617+0000 7f43d08b9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43cc070110 0x7f43cc1356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:44.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.617+0000 7f43d08b9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43cc070110 0x7f43cc1356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37970/0 (socket says 172.21.15.60:37970) 2023-08-24T10:09:44.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.617+0000 7f43d10ba640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f43cc071e40 0x7f43cc096800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41196/0 (socket says 172.21.15.60:41196) 2023-08-24T10:09:44.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.617+0000 7f43d08b9640 1 -- 172.21.15.60:0/2489353472 learned_addr learned my addr 172.21.15.60:0/2489353472 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:44.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.618+0000 7f43d08b9640 1 -- 172.21.15.60:0/2489353472 >> v1:172.21.15.60:6789/0 conn(0x7f43cc071e40 legacy=0x7f43cc096800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:44.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.618+0000 7f43d08b9640 1 -- 172.21.15.60:0/2489353472 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc09a280 con 0x7f43cc070110 2023-08-24T10:09:44.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.618+0000 7f43d08b9640 1 --2- 172.21.15.60:0/2489353472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43cc070110 0x7f43cc1356a0 crc :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65649a604a156fff server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:44.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.618+0000 7f43cbfff640 1 -- 172.21.15.60:0/2489353472 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f43c0009dd0 con 0x7f43cc070110 2023-08-24T10:09:44.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.618+0000 7f43cbfff640 1 -- 172.21.15.60:0/2489353472 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f43c000e230 con 0x7f43cc070110 2023-08-24T10:09:44.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.618+0000 7f43cbfff640 1 -- 172.21.15.60:0/2489353472 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f43c000e4d0 con 0x7f43cc070110 2023-08-24T10:09:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.619+0000 7f43d331e640 1 -- 172.21.15.60:0/2489353472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43cc070110 msgr2=0x7f43cc1356a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:44.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.619+0000 7f43d331e640 1 --2- 172.21.15.60:0/2489353472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43cc070110 0x7f43cc1356a0 crc :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:44.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.619+0000 7f43d331e640 1 -- 172.21.15.60:0/2489353472 shutdown_connections 2023-08-24T10:09:44.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.619+0000 7f43d331e640 1 --2- 172.21.15.60:0/2489353472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43cc070110 0x7f43cc1356a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:44.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.619+0000 7f43d331e640 1 -- 172.21.15.60:0/2489353472 >> 172.21.15.60:0/2489353472 conn(0x7f43cc09e110 msgr2=0x7f43cc09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:44.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.619+0000 7f43d331e640 1 -- 172.21.15.60:0/2489353472 shutdown_connections 2023-08-24T10:09:44.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.619+0000 7f43d331e640 1 -- 172.21.15.60:0/2489353472 wait complete. 2023-08-24T10:09:44.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.619+0000 7f43d331e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.620+0000 7f43d331e640 1 Processor -- start 2023-08-24T10:09:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.620+0000 7f43d331e640 1 -- start start 2023-08-24T10:09:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.620+0000 7f43d331e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43cc070110 0x7f43cc093460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.620+0000 7f43d331e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc08ff10 con 0x7f43cc070110 2023-08-24T10:09:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.621+0000 7f43d10ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43cc070110 0x7f43cc093460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.621+0000 7f43d10ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43cc070110 0x7f43cc093460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37986/0 (socket says 172.21.15.60:37986) 2023-08-24T10:09:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.621+0000 7f43d10ba640 1 -- 172.21.15.60:0/2263048139 learned_addr learned my addr 172.21.15.60:0/2263048139 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.621+0000 7f43d10ba640 1 -- 172.21.15.60:0/2263048139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc090680 con 0x7f43cc070110 2023-08-24T10:09:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.621+0000 7f43d10ba640 1 --2- 172.21.15.60:0/2263048139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43cc070110 0x7f43cc093460 crc :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.621+0000 7f43c9ffb640 1 -- 172.21.15.60:0/2263048139 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f43c40071e0 con 0x7f43cc070110 2023-08-24T10:09:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.621+0000 7f43c9ffb640 1 -- 172.21.15.60:0/2263048139 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f43c4007340 con 0x7f43cc070110 2023-08-24T10:09:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.621+0000 7f43d331e640 1 -- 172.21.15.60:0/2263048139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43cc092e80 con 0x7f43cc070110 2023-08-24T10:09:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.621+0000 7f43c9ffb640 1 -- 172.21.15.60:0/2263048139 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f43c40075e0 con 0x7f43cc070110 2023-08-24T10:09:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.621+0000 7f43d331e640 1 -- 172.21.15.60:0/2263048139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43cc092b70 con 0x7f43cc070110 2023-08-24T10:09:44.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.622+0000 7f43c9ffb640 1 -- 172.21.15.60:0/2263048139 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f43c40057e0 con 0x7f43cc070110 2023-08-24T10:09:44.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.622+0000 7f43c9ffb640 1 --2- 172.21.15.60:0/2263048139 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f43b8037180 0x7f43b8039690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:44.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.622+0000 7f43c9ffb640 1 -- 172.21.15.60:0/2263048139 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f43c40463f0 con 0x7f43cc070110 2023-08-24T10:09:44.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.622+0000 7f43d331e640 1 -- 172.21.15.60:0/2263048139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43cc061800 con 0x7f43cc070110 2023-08-24T10:09:44.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.623+0000 7f43d08b9640 1 --2- 172.21.15.60:0/2263048139 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f43b8037180 0x7f43b8039690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:44.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.623+0000 7f43d08b9640 1 --2- 172.21.15.60:0/2263048139 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f43b8037180 0x7f43b8039690 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:44.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.627+0000 7f43c9ffb640 1 -- 172.21.15.60:0/2263048139 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f43c400d330 con 0x7f43cc070110 2023-08-24T10:09:44.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.800+0000 7f43d331e640 1 -- 172.21.15.60:0/2263048139 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43cc0932f0 con 0x7f43b8037180 2023-08-24T10:09:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.802+0000 7f43c9ffb640 1 -- 172.21.15.60:0/2263048139 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f43cc0932f0 con 0x7f43b8037180 2023-08-24T10:09:44.803 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:44.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.803+0000 7f43d331e640 1 -- 172.21.15.60:0/2263048139 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f43b8037180 msgr2=0x7f43b8039690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:44.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.803+0000 7f43d331e640 1 --2- 172.21.15.60:0/2263048139 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f43b8037180 0x7f43b8039690 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:44.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.803+0000 7f43d331e640 1 -- 172.21.15.60:0/2263048139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43cc070110 msgr2=0x7f43cc093460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:44.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.803+0000 7f43d331e640 1 --2- 172.21.15.60:0/2263048139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43cc070110 0x7f43cc093460 crc :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:44.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.803+0000 7f43d331e640 1 -- 172.21.15.60:0/2263048139 shutdown_connections 2023-08-24T10:09:44.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.803+0000 7f43d331e640 1 --2- 172.21.15.60:0/2263048139 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f43b8037180 0x7f43b8039690 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:44.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.803+0000 7f43d331e640 1 --2- 172.21.15.60:0/2263048139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43cc070110 0x7f43cc093460 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:44.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.803+0000 7f43d331e640 1 -- 172.21.15.60:0/2263048139 >> 172.21.15.60:0/2263048139 conn(0x7f43cc09e110 msgr2=0x7f43cc098490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:44.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.803+0000 7f43d331e640 1 -- 172.21.15.60:0/2263048139 shutdown_connections 2023-08-24T10:09:44.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:44.803+0000 7f43d331e640 1 -- 172.21.15.60:0/2263048139 wait complete. 2023-08-24T10:09:44.815 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2023-08-24T10:09:43.550307+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:47.817 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.936+0000 7feb4831e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:47.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.940+0000 7feb4831e640 1 Processor -- start 2023-08-24T10:09:47.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.940+0000 7feb4831e640 1 -- start start 2023-08-24T10:09:47.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.941+0000 7feb4831e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feb40132630 0x7feb40134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:47.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.941+0000 7feb4831e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb4006e320 con 0x7feb40096860 2023-08-24T10:09:47.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.941+0000 7feb4831e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb4006db70 con 0x7feb40132630 2023-08-24T10:09:47.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.941+0000 7feb458b9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feb40132630 0x7feb40134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:47.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.941+0000 7feb458b9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feb40132630 0x7feb40134a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36590/0 (socket says 172.21.15.60:36590) 2023-08-24T10:09:47.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.941+0000 7feb460ba640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7feb40096860 0x7feb40096c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48568/0 (socket says 172.21.15.60:48568) 2023-08-24T10:09:47.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.941+0000 7feb458b9640 1 -- 172.21.15.60:0/340592118 learned_addr learned my addr 172.21.15.60:0/340592118 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:47.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.941+0000 7feb458b9640 1 -- 172.21.15.60:0/340592118 >> v1:172.21.15.60:6789/0 conn(0x7feb40096860 legacy=0x7feb40096c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:47.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.941+0000 7feb458b9640 1 -- 172.21.15.60:0/340592118 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4006cef0 con 0x7feb40132630 2023-08-24T10:09:47.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.942+0000 7feb458b9640 1 --2- 172.21.15.60:0/340592118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40132630 0x7feb40134a50 crc :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a7ab66112f0f0c server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:47.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.942+0000 7feb450b8640 1 -- 172.21.15.60:0/340592118 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7feb30009dd0 con 0x7feb40132630 2023-08-24T10:09:47.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.942+0000 7feb450b8640 1 -- 172.21.15.60:0/340592118 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7feb3000e230 con 0x7feb40132630 2023-08-24T10:09:47.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.942+0000 7feb450b8640 1 -- 172.21.15.60:0/340592118 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7feb3000e4d0 con 0x7feb40132630 2023-08-24T10:09:47.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.942+0000 7feb4831e640 1 -- 172.21.15.60:0/340592118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40132630 msgr2=0x7feb40134a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:47.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.942+0000 7feb4831e640 1 --2- 172.21.15.60:0/340592118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40132630 0x7feb40134a50 crc :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:47.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.943+0000 7feb4831e640 1 -- 172.21.15.60:0/340592118 shutdown_connections 2023-08-24T10:09:47.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.943+0000 7feb4831e640 1 --2- 172.21.15.60:0/340592118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40132630 0x7feb40134a50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:47.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.943+0000 7feb4831e640 1 -- 172.21.15.60:0/340592118 >> 172.21.15.60:0/340592118 conn(0x7feb4009e110 msgr2=0x7feb4009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:47.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.943+0000 7feb4831e640 1 -- 172.21.15.60:0/340592118 shutdown_connections 2023-08-24T10:09:47.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.943+0000 7feb4831e640 1 -- 172.21.15.60:0/340592118 wait complete. 2023-08-24T10:09:47.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.943+0000 7feb4831e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:47.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.944+0000 7feb4831e640 1 Processor -- start 2023-08-24T10:09:47.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.944+0000 7feb4831e640 1 -- start start 2023-08-24T10:09:47.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.944+0000 7feb4831e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40096860 0x7feb401c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:47.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.945+0000 7feb4831e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb40098ce0 con 0x7feb40096860 2023-08-24T10:09:47.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.945+0000 7feb460ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40096860 0x7feb401c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:47.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.945+0000 7feb460ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40096860 0x7feb401c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36606/0 (socket says 172.21.15.60:36606) 2023-08-24T10:09:47.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.945+0000 7feb460ba640 1 -- 172.21.15.60:0/3226434315 learned_addr learned my addr 172.21.15.60:0/3226434315 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:47.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.945+0000 7feb460ba640 1 -- 172.21.15.60:0/3226434315 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4012b2a0 con 0x7feb40096860 2023-08-24T10:09:47.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.945+0000 7feb460ba640 1 --2- 172.21.15.60:0/3226434315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40096860 0x7feb401c52a0 crc :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:47.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.945+0000 7feb36ffd640 1 -- 172.21.15.60:0/3226434315 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7feb3c0071e0 con 0x7feb40096860 2023-08-24T10:09:47.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.945+0000 7feb36ffd640 1 -- 172.21.15.60:0/3226434315 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7feb3c007340 con 0x7feb40096860 2023-08-24T10:09:47.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.945+0000 7feb4831e640 1 -- 172.21.15.60:0/3226434315 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4012b9e0 con 0x7feb40096860 2023-08-24T10:09:47.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.945+0000 7feb36ffd640 1 -- 172.21.15.60:0/3226434315 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7feb3c0075e0 con 0x7feb40096860 2023-08-24T10:09:47.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.945+0000 7feb4831e640 1 -- 172.21.15.60:0/3226434315 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb401c4e70 con 0x7feb40096860 2023-08-24T10:09:47.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.946+0000 7feb36ffd640 1 -- 172.21.15.60:0/3226434315 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7feb3c0057e0 con 0x7feb40096860 2023-08-24T10:09:47.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.946+0000 7feb36ffd640 1 --2- 172.21.15.60:0/3226434315 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7feb24037130 0x7feb24039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:47.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.946+0000 7feb36ffd640 1 -- 172.21.15.60:0/3226434315 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7feb3c046340 con 0x7feb40096860 2023-08-24T10:09:47.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.946+0000 7feb4831e640 1 -- 172.21.15.60:0/3226434315 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb40061800 con 0x7feb40096860 2023-08-24T10:09:47.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.947+0000 7feb458b9640 1 --2- 172.21.15.60:0/3226434315 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7feb24037130 0x7feb24039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:47.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.947+0000 7feb458b9640 1 --2- 172.21.15.60:0/3226434315 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7feb24037130 0x7feb24039640 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:47.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:47.951+0000 7feb36ffd640 1 -- 172.21.15.60:0/3226434315 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7feb3c00d330 con 0x7feb40096860 2023-08-24T10:09:48.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.125+0000 7feb4831e640 1 -- 172.21.15.60:0/3226434315 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feb4006d190 con 0x7feb24037130 2023-08-24T10:09:48.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.127+0000 7feb36ffd640 1 -- 172.21.15.60:0/3226434315 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7feb30009df0 con 0x7feb24037130 2023-08-24T10:09:48.128 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:48.129 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:48.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.128+0000 7feb4831e640 1 -- 172.21.15.60:0/3226434315 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7feb24037130 msgr2=0x7feb24039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:48.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.128+0000 7feb4831e640 1 --2- 172.21.15.60:0/3226434315 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7feb24037130 0x7feb24039640 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.128+0000 7feb4831e640 1 -- 172.21.15.60:0/3226434315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40096860 msgr2=0x7feb401c52a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.128+0000 7feb4831e640 1 --2- 172.21.15.60:0/3226434315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40096860 0x7feb401c52a0 crc :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.128+0000 7feb4831e640 1 -- 172.21.15.60:0/3226434315 shutdown_connections 2023-08-24T10:09:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.128+0000 7feb4831e640 1 --2- 172.21.15.60:0/3226434315 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7feb24037130 0x7feb24039640 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.128+0000 7feb4831e640 1 --2- 172.21.15.60:0/3226434315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb40096860 0x7feb401c52a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.128+0000 7feb4831e640 1 -- 172.21.15.60:0/3226434315 >> 172.21.15.60:0/3226434315 conn(0x7feb4009e110 msgr2=0x7feb400950c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.128+0000 7feb4831e640 1 -- 172.21.15.60:0/3226434315 shutdown_connections 2023-08-24T10:09:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.128+0000 7feb4831e640 1 -- 172.21.15.60:0/3226434315 wait complete. 2023-08-24T10:09:48.141 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2023-08-24T10:09:47.551260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:48.142 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:48.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.249+0000 7f01b1c30640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:48.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.253+0000 7f01b1c30640 1 Processor -- start 2023-08-24T10:09:48.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.253+0000 7f01b1c30640 1 -- start start 2023-08-24T10:09:48.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.253+0000 7f01b1c30640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f01ac135980 0x7f01ac137da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:48.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.253+0000 7f01b1c30640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ac132610 con 0x7f01ac132b50 2023-08-24T10:09:48.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.253+0000 7f01b1c30640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ac09c260 con 0x7f01ac135980 2023-08-24T10:09:48.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.253+0000 7f01ab7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f01ac132b50 0x7f01ac132f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48580/0 (socket says 172.21.15.60:48580) 2023-08-24T10:09:48.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.253+0000 7f01ab7fe640 1 -- 172.21.15.60:0/1450575965 learned_addr learned my addr 172.21.15.60:0/1450575965 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:09:48.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.254+0000 7f01aaffd640 1 --2- 172.21.15.60:0/1450575965 >> v2:172.21.15.60:3300/0 conn(0x7f01ac135980 0x7f01ac137da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:48.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.254+0000 7f01aaffd640 1 -- 172.21.15.60:0/1450575965 >> v1:172.21.15.60:6789/0 conn(0x7f01ac132b50 legacy=0x7f01ac132f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:48.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.254+0000 7f01aaffd640 1 -- 172.21.15.60:0/1450575965 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac132840 con 0x7f01ac135980 2023-08-24T10:09:48.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.254+0000 7f01aaffd640 1 --2- 172.21.15.60:0/1450575965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01ac135980 0x7f01ac137da0 crc :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e16d287609093f server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:48.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.254+0000 7f01aa7fc640 1 -- 172.21.15.60:0/1450575965 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f01a0009ce0 con 0x7f01ac135980 2023-08-24T10:09:48.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.254+0000 7f01aa7fc640 1 -- 172.21.15.60:0/1450575965 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f01a0009e40 con 0x7f01ac135980 2023-08-24T10:09:48.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.254+0000 7f01aa7fc640 1 -- 172.21.15.60:0/1450575965 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f01a000e360 con 0x7f01ac135980 2023-08-24T10:09:48.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.255+0000 7f01b1c30640 1 -- 172.21.15.60:0/1450575965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01ac135980 msgr2=0x7f01ac137da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:48.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.255+0000 7f01b1c30640 1 --2- 172.21.15.60:0/1450575965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01ac135980 0x7f01ac137da0 crc :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:48.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.255+0000 7f01b1c30640 1 -- 172.21.15.60:0/1450575965 shutdown_connections 2023-08-24T10:09:48.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.255+0000 7f01b1c30640 1 --2- 172.21.15.60:0/1450575965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01ac135980 0x7f01ac137da0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:48.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.255+0000 7f01b1c30640 1 -- 172.21.15.60:0/1450575965 >> 172.21.15.60:0/1450575965 conn(0x7f01ac0a26a0 msgr2=0x7f01ac0a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:48.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.255+0000 7f01b1c30640 1 -- 172.21.15.60:0/1450575965 shutdown_connections 2023-08-24T10:09:48.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.255+0000 7f01b1c30640 1 -- 172.21.15.60:0/1450575965 wait complete. 2023-08-24T10:09:48.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.255+0000 7f01b1c30640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:48.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.256+0000 7f01b1c30640 1 Processor -- start 2023-08-24T10:09:48.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.256+0000 7f01b1c30640 1 -- start start 2023-08-24T10:09:48.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.257+0000 7f01b1c30640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01ac132b50 0x7f01ac0a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:48.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.257+0000 7f01b1c30640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac0a1b60 con 0x7f01ac132b50 2023-08-24T10:09:48.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.257+0000 7f01ab7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01ac132b50 0x7f01ac0a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:48.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.257+0000 7f01ab7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01ac132b50 0x7f01ac0a0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36630/0 (socket says 172.21.15.60:36630) 2023-08-24T10:09:48.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.257+0000 7f01ab7fe640 1 -- 172.21.15.60:0/1340394085 learned_addr learned my addr 172.21.15.60:0/1340394085 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:48.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.257+0000 7f01ab7fe640 1 -- 172.21.15.60:0/1340394085 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac09fab0 con 0x7f01ac132b50 2023-08-24T10:09:48.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.257+0000 7f01ab7fe640 1 --2- 172.21.15.60:0/1340394085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01ac132b50 0x7f01ac0a0000 crc :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:48.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.258+0000 7f0193fff640 1 -- 172.21.15.60:0/1340394085 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f019c0059d0 con 0x7f01ac132b50 2023-08-24T10:09:48.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.258+0000 7f0193fff640 1 -- 172.21.15.60:0/1340394085 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f019c005b30 con 0x7f01ac132b50 2023-08-24T10:09:48.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.258+0000 7f0193fff640 1 -- 172.21.15.60:0/1340394085 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f019c005dd0 con 0x7f01ac132b50 2023-08-24T10:09:48.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.258+0000 7f01b1c30640 1 -- 172.21.15.60:0/1340394085 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ac06a240 con 0x7f01ac132b50 2023-08-24T10:09:48.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.258+0000 7f01b1c30640 1 -- 172.21.15.60:0/1340394085 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ac11ac50 con 0x7f01ac132b50 2023-08-24T10:09:48.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.258+0000 7f0193fff640 1 -- 172.21.15.60:0/1340394085 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f019c0070b0 con 0x7f01ac132b50 2023-08-24T10:09:48.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.259+0000 7f01b1c30640 1 -- 172.21.15.60:0/1340394085 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ac061800 con 0x7f01ac132b50 2023-08-24T10:09:48.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.259+0000 7f0193fff640 1 --2- 172.21.15.60:0/1340394085 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0198037130 0x7f0198039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:48.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.259+0000 7f0193fff640 1 -- 172.21.15.60:0/1340394085 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f019c046170 con 0x7f01ac132b50 2023-08-24T10:09:48.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.259+0000 7f01aaffd640 1 --2- 172.21.15.60:0/1340394085 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0198037130 0x7f0198039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:48.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.260+0000 7f01aaffd640 1 --2- 172.21.15.60:0/1340394085 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0198037130 0x7f0198039640 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:48.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.263+0000 7f0193fff640 1 -- 172.21.15.60:0/1340394085 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f019c00d330 con 0x7f01ac132b50 2023-08-24T10:09:48.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.438+0000 7f01b1c30640 1 -- 172.21.15.60:0/1340394085 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f01ac09f940 con 0x7f0198037130 2023-08-24T10:09:48.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.440+0000 7f0193fff640 1 -- 172.21.15.60:0/1340394085 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f01ac09f940 con 0x7f0198037130 2023-08-24T10:09:48.442 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:48.442 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:48.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.441+0000 7f01b1c30640 1 -- 172.21.15.60:0/1340394085 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0198037130 msgr2=0x7f0198039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.441+0000 7f01b1c30640 1 --2- 172.21.15.60:0/1340394085 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0198037130 0x7f0198039640 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.441+0000 7f01b1c30640 1 -- 172.21.15.60:0/1340394085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01ac132b50 msgr2=0x7f01ac0a0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.441+0000 7f01b1c30640 1 --2- 172.21.15.60:0/1340394085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01ac132b50 0x7f01ac0a0000 crc :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.441+0000 7f01b1c30640 1 -- 172.21.15.60:0/1340394085 shutdown_connections 2023-08-24T10:09:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.441+0000 7f01b1c30640 1 --2- 172.21.15.60:0/1340394085 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0198037130 0x7f0198039640 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.441+0000 7f01b1c30640 1 --2- 172.21.15.60:0/1340394085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01ac132b50 0x7f01ac0a0000 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:48.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.441+0000 7f01b1c30640 1 -- 172.21.15.60:0/1340394085 >> 172.21.15.60:0/1340394085 conn(0x7f01ac0a26a0 msgr2=0x7f01ac1baaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:48.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.442+0000 7f01b1c30640 1 -- 172.21.15.60:0/1340394085 shutdown_connections 2023-08-24T10:09:48.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.442+0000 7f01b1c30640 1 -- 172.21.15.60:0/1340394085 wait complete. 2023-08-24T10:09:48.454 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2023-08-24T10:09:47.551260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:48.455 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:09:48.455 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:48.455 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:09:48.456 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:48.456 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:09:48.456 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:48.456 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:09:48.457 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:48.457 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:09:48.457 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:48.457 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:09:48.458 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:48.458 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:09:48.458 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:48.458 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:09:48.459 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:48.459 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:09:48.459 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:48.459 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:09:48.465 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~281s 2023-08-24T10:09:48.465 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~281s 2023-08-24T10:09:48.465 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~281s 2023-08-24T10:09:48.465 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~281s 2023-08-24T10:09:48.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.566+0000 7f54df856640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:48.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.569+0000 7f54df856640 1 Processor -- start 2023-08-24T10:09:48.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.569+0000 7f54df856640 1 -- start start 2023-08-24T10:09:48.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.570+0000 7f54df856640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f54d8070c30 0x7f54d8071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.570+0000 7f54df856640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54d806dfc0 con 0x7f54d80990b0 2023-08-24T10:09:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.570+0000 7f54df856640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d806d9b0 con 0x7f54d8070c30 2023-08-24T10:09:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.570+0000 7f54dd5f2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f54d8070c30 0x7f54d8071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.570+0000 7f54dd5f2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f54d8070c30 0x7f54d8071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36640/0 (socket says 172.21.15.60:36640) 2023-08-24T10:09:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.570+0000 7f54dd5f2640 1 -- 172.21.15.60:0/2517481579 learned_addr learned my addr 172.21.15.60:0/2517481579 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.570+0000 7f54dd5f2640 1 -- 172.21.15.60:0/2517481579 >> v1:172.21.15.60:6789/0 conn(0x7f54d80990b0 legacy=0x7f54d80994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.570+0000 7f54dd5f2640 1 -- 172.21.15.60:0/2517481579 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d806e6e0 con 0x7f54d8070c30 2023-08-24T10:09:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.571+0000 7f54dd5f2640 1 --2- 172.21.15.60:0/2517481579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54d8070c30 0x7f54d8071040 crc :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bab85f8854e358 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.571+0000 7f54cffff640 1 -- 172.21.15.60:0/2517481579 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f54d400e040 con 0x7f54d8070c30 2023-08-24T10:09:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.571+0000 7f54cffff640 1 -- 172.21.15.60:0/2517481579 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f54d40091e0 con 0x7f54d8070c30 2023-08-24T10:09:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.571+0000 7f54cffff640 1 -- 172.21.15.60:0/2517481579 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f54d4007730 con 0x7f54d8070c30 2023-08-24T10:09:48.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.571+0000 7f54df856640 1 -- 172.21.15.60:0/2517481579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54d8070c30 msgr2=0x7f54d8071040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:48.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.571+0000 7f54df856640 1 --2- 172.21.15.60:0/2517481579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54d8070c30 0x7f54d8071040 crc :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:48.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.571+0000 7f54df856640 1 -- 172.21.15.60:0/2517481579 shutdown_connections 2023-08-24T10:09:48.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.571+0000 7f54df856640 1 --2- 172.21.15.60:0/2517481579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54d8070c30 0x7f54d8071040 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:48.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.571+0000 7f54df856640 1 -- 172.21.15.60:0/2517481579 >> 172.21.15.60:0/2517481579 conn(0x7f54d809e110 msgr2=0x7f54d809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:48.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.572+0000 7f54df856640 1 -- 172.21.15.60:0/2517481579 shutdown_connections 2023-08-24T10:09:48.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.572+0000 7f54df856640 1 -- 172.21.15.60:0/2517481579 wait complete. 2023-08-24T10:09:48.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.572+0000 7f54df856640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:48.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.573+0000 7f54df856640 1 Processor -- start 2023-08-24T10:09:48.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.573+0000 7f54df856640 1 -- start start 2023-08-24T10:09:48.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.573+0000 7f54df856640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54d8070c30 0x7f54d81c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:48.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.573+0000 7f54df856640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d812b3e0 con 0x7f54d8070c30 2023-08-24T10:09:48.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.573+0000 7f54dd5f2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54d8070c30 0x7f54d81c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:48.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.573+0000 7f54dd5f2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54d8070c30 0x7f54d81c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36648/0 (socket says 172.21.15.60:36648) 2023-08-24T10:09:48.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.573+0000 7f54dd5f2640 1 -- 172.21.15.60:0/2381871512 learned_addr learned my addr 172.21.15.60:0/2381871512 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:48.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.574+0000 7f54dd5f2640 1 -- 172.21.15.60:0/2381871512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d81c0930 con 0x7f54d8070c30 2023-08-24T10:09:48.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.574+0000 7f54dd5f2640 1 --2- 172.21.15.60:0/2381871512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54d8070c30 0x7f54d81c0fa0 crc :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:48.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.574+0000 7f54cdffb640 1 -- 172.21.15.60:0/2381871512 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f54d400e040 con 0x7f54d8070c30 2023-08-24T10:09:48.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.574+0000 7f54cdffb640 1 -- 172.21.15.60:0/2381871512 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f54d4012720 con 0x7f54d8070c30 2023-08-24T10:09:48.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.574+0000 7f54cdffb640 1 -- 172.21.15.60:0/2381871512 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f54d40129c0 con 0x7f54d8070c30 2023-08-24T10:09:48.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.574+0000 7f54df856640 1 -- 172.21.15.60:0/2381871512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d81c0c00 con 0x7f54d8070c30 2023-08-24T10:09:48.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.574+0000 7f54df856640 1 -- 172.21.15.60:0/2381871512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d812c980 con 0x7f54d8070c30 2023-08-24T10:09:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.575+0000 7f54cdffb640 1 -- 172.21.15.60:0/2381871512 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f54d4011040 con 0x7f54d8070c30 2023-08-24T10:09:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.575+0000 7f54cdffb640 1 --2- 172.21.15.60:0/2381871512 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f54bc037130 0x7f54bc039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.575+0000 7f54cdffb640 1 -- 172.21.15.60:0/2381871512 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f54d4047a30 con 0x7f54d8070c30 2023-08-24T10:09:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.575+0000 7f54df856640 1 -- 172.21.15.60:0/2381871512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d812dd10 con 0x7f54d8070c30 2023-08-24T10:09:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.575+0000 7f54dcdf1640 1 --2- 172.21.15.60:0/2381871512 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f54bc037130 0x7f54bc039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.576+0000 7f54dcdf1640 1 --2- 172.21.15.60:0/2381871512 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f54bc037130 0x7f54bc039640 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:48.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.580+0000 7f54cdffb640 1 -- 172.21.15.60:0/2381871512 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f54d4015970 con 0x7f54d8070c30 2023-08-24T10:09:48.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.822+0000 7f54df856640 1 -- 172.21.15.60:0/2381871512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f54d812c1b0 con 0x7f54d8070c30 2023-08-24T10:09:48.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.823+0000 7f54cdffb640 1 -- 172.21.15.60:0/2381871512 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f54d4013e60 con 0x7f54d8070c30 2023-08-24T10:09:48.825 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:48.825 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":299,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:09:48.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.824+0000 7f54df856640 1 -- 172.21.15.60:0/2381871512 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f54bc037130 msgr2=0x7f54bc039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:48.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.824+0000 7f54df856640 1 --2- 172.21.15.60:0/2381871512 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f54bc037130 0x7f54bc039640 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:48.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.824+0000 7f54df856640 1 -- 172.21.15.60:0/2381871512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54d8070c30 msgr2=0x7f54d81c0fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:48.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.824+0000 7f54df856640 1 --2- 172.21.15.60:0/2381871512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54d8070c30 0x7f54d81c0fa0 crc :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:48.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.824+0000 7f54df856640 1 -- 172.21.15.60:0/2381871512 shutdown_connections 2023-08-24T10:09:48.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.824+0000 7f54df856640 1 --2- 172.21.15.60:0/2381871512 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f54bc037130 0x7f54bc039640 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:48.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.824+0000 7f54df856640 1 --2- 172.21.15.60:0/2381871512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54d8070c30 0x7f54d81c0fa0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:48.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.824+0000 7f54df856640 1 -- 172.21.15.60:0/2381871512 >> 172.21.15.60:0/2381871512 conn(0x7f54d809e110 msgr2=0x7f54d81b75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:48.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.825+0000 7f54df856640 1 -- 172.21.15.60:0/2381871512 shutdown_connections 2023-08-24T10:09:48.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.825+0000 7f54df856640 1 -- 172.21.15.60:0/2381871512 wait complete. 2023-08-24T10:09:48.837 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:09:48.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:48.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.943+0000 7f625779e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:48.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.947+0000 7f625779e640 1 Processor -- start 2023-08-24T10:09:48.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.947+0000 7f625779e640 1 -- start start 2023-08-24T10:09:48.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.948+0000 7f625779e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6250096860 0x7f6250096c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:48.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.948+0000 7f625779e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62500971b0 con 0x7f62500990b0 2023-08-24T10:09:48.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.948+0000 7f625779e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f625006db70 con 0x7f6250096860 2023-08-24T10:09:48.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.948+0000 7f625553a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6250096860 0x7f6250096c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:48.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.948+0000 7f625553a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6250096860 0x7f6250096c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36654/0 (socket says 172.21.15.60:36654) 2023-08-24T10:09:48.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.948+0000 7f6254d39640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f62500990b0 0x7f62500994c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48602/0 (socket says 172.21.15.60:48602) 2023-08-24T10:09:48.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.948+0000 7f625553a640 1 -- 172.21.15.60:0/3589026806 learned_addr learned my addr 172.21.15.60:0/3589026806 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:48.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.948+0000 7f625553a640 1 -- 172.21.15.60:0/3589026806 >> v1:172.21.15.60:6789/0 conn(0x7f62500990b0 legacy=0x7f62500994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:48.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.948+0000 7f625553a640 1 -- 172.21.15.60:0/3589026806 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625006e320 con 0x7f6250096860 2023-08-24T10:09:48.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.948+0000 7f625553a640 1 --2- 172.21.15.60:0/3589026806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6250096860 0x7f6250096c70 crc :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53323a37c050d820 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:48.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.948+0000 7f6247fff640 1 -- 172.21.15.60:0/3589026806 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f624c009dd0 con 0x7f6250096860 2023-08-24T10:09:48.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.948+0000 7f6247fff640 1 -- 172.21.15.60:0/3589026806 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f624c00e230 con 0x7f6250096860 2023-08-24T10:09:48.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.948+0000 7f6247fff640 1 -- 172.21.15.60:0/3589026806 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f624c00e4d0 con 0x7f6250096860 2023-08-24T10:09:48.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.949+0000 7f625779e640 1 -- 172.21.15.60:0/3589026806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6250096860 msgr2=0x7f6250096c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:48.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.949+0000 7f625779e640 1 --2- 172.21.15.60:0/3589026806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6250096860 0x7f6250096c70 crc :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:48.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.949+0000 7f625779e640 1 -- 172.21.15.60:0/3589026806 shutdown_connections 2023-08-24T10:09:48.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.949+0000 7f625779e640 1 --2- 172.21.15.60:0/3589026806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6250096860 0x7f6250096c70 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:48.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.949+0000 7f625779e640 1 -- 172.21.15.60:0/3589026806 >> 172.21.15.60:0/3589026806 conn(0x7f625009e110 msgr2=0x7f625009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:48.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.949+0000 7f625779e640 1 -- 172.21.15.60:0/3589026806 shutdown_connections 2023-08-24T10:09:48.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.949+0000 7f625779e640 1 -- 172.21.15.60:0/3589026806 wait complete. 2023-08-24T10:09:48.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.949+0000 7f625779e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:48.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.950+0000 7f625779e640 1 Processor -- start 2023-08-24T10:09:48.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.950+0000 7f625779e640 1 -- start start 2023-08-24T10:09:48.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.951+0000 7f625779e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6250096860 0x7f62501c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:48.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.951+0000 7f625779e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62501c4e10 con 0x7f6250096860 2023-08-24T10:09:48.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.951+0000 7f625553a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6250096860 0x7f62501c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:48.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.951+0000 7f625553a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6250096860 0x7f62501c51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36668/0 (socket says 172.21.15.60:36668) 2023-08-24T10:09:48.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.951+0000 7f625553a640 1 -- 172.21.15.60:0/197181356 learned_addr learned my addr 172.21.15.60:0/197181356 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:48.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.951+0000 7f625553a640 1 -- 172.21.15.60:0/197181356 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625012bd60 con 0x7f6250096860 2023-08-24T10:09:48.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.951+0000 7f625553a640 1 --2- 172.21.15.60:0/197181356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6250096860 0x7f62501c51d0 crc :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:48.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.952+0000 7f6245ffb640 1 -- 172.21.15.60:0/197181356 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f624c014410 con 0x7f6250096860 2023-08-24T10:09:48.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.952+0000 7f6245ffb640 1 -- 172.21.15.60:0/197181356 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f624c014570 con 0x7f6250096860 2023-08-24T10:09:48.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.952+0000 7f625779e640 1 -- 172.21.15.60:0/197181356 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625012ae60 con 0x7f6250096860 2023-08-24T10:09:48.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.952+0000 7f6245ffb640 1 -- 172.21.15.60:0/197181356 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f624c014810 con 0x7f6250096860 2023-08-24T10:09:48.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.952+0000 7f625779e640 1 -- 172.21.15.60:0/197181356 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62500989e0 con 0x7f6250096860 2023-08-24T10:09:48.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.953+0000 7f6245ffb640 1 -- 172.21.15.60:0/197181356 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f624c0149c0 con 0x7f6250096860 2023-08-24T10:09:48.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.953+0000 7f6245ffb640 1 --2- 172.21.15.60:0/197181356 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6234037130 0x7f6234039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:48.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.953+0000 7f6245ffb640 1 -- 172.21.15.60:0/197181356 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f624c019040 con 0x7f6250096860 2023-08-24T10:09:48.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.953+0000 7f625779e640 1 -- 172.21.15.60:0/197181356 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f625012a400 con 0x7f6250096860 2023-08-24T10:09:48.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.953+0000 7f6254d39640 1 --2- 172.21.15.60:0/197181356 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6234037130 0x7f6234039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:48.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.953+0000 7f6254d39640 1 --2- 172.21.15.60:0/197181356 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6234037130 0x7f6234039640 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:48.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:48.958+0000 7f6245ffb640 1 -- 172.21.15.60:0/197181356 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f624c00f8f0 con 0x7f6250096860 2023-08-24T10:09:49.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:49.131+0000 7f625779e640 1 -- 172.21.15.60:0/197181356 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f625012bb00 con 0x7f6234037130 2023-08-24T10:09:49.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:49.133+0000 7f6245ffb640 1 -- 172.21.15.60:0/197181356 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f62400070b0 con 0x7f6234037130 2023-08-24T10:09:49.135 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:49.135 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:49.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:49.135+0000 7f625779e640 1 -- 172.21.15.60:0/197181356 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6234037130 msgr2=0x7f6234039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:49.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:49.135+0000 7f625779e640 1 --2- 172.21.15.60:0/197181356 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6234037130 0x7f6234039640 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:49.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:49.135+0000 7f625779e640 1 -- 172.21.15.60:0/197181356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6250096860 msgr2=0x7f62501c51d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:49.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:49.135+0000 7f625779e640 1 --2- 172.21.15.60:0/197181356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6250096860 0x7f62501c51d0 crc :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:49.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:49.135+0000 7f625779e640 1 -- 172.21.15.60:0/197181356 shutdown_connections 2023-08-24T10:09:49.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:49.135+0000 7f625779e640 1 --2- 172.21.15.60:0/197181356 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6234037130 0x7f6234039640 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:49.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:49.135+0000 7f625779e640 1 --2- 172.21.15.60:0/197181356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6250096860 0x7f62501c51d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:49.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:49.135+0000 7f625779e640 1 -- 172.21.15.60:0/197181356 >> 172.21.15.60:0/197181356 conn(0x7f625009e110 msgr2=0x7f625009bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:49.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:49.135+0000 7f625779e640 1 -- 172.21.15.60:0/197181356 shutdown_connections 2023-08-24T10:09:49.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:49.135+0000 7f625779e640 1 -- 172.21.15.60:0/197181356 wait complete. 2023-08-24T10:09:49.147 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2023-08-24T10:09:47.551260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:52.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:52.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.269+0000 7f9a2d99e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:52.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.273+0000 7f9a2d99e640 1 Processor -- start 2023-08-24T10:09:52.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.273+0000 7f9a2d99e640 1 -- start start 2023-08-24T10:09:52.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.274+0000 7f9a2d99e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9a28070110 0x7f9a281356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:52.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.274+0000 7f9a2d99e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a28096f10 con 0x7f9a28071e40 2023-08-24T10:09:52.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.274+0000 7f9a2d99e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a280977f0 con 0x7f9a28070110 2023-08-24T10:09:52.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.274+0000 7f9a1e7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9a28070110 0x7f9a281356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:52.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.274+0000 7f9a1e7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9a28070110 0x7f9a281356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36684/0 (socket says 172.21.15.60:36684) 2023-08-24T10:09:52.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.274+0000 7f9a1e7fc640 1 -- 172.21.15.60:0/1887603518 learned_addr learned my addr 172.21.15.60:0/1887603518 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:52.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.274+0000 7f9a1e7fc640 1 -- 172.21.15.60:0/1887603518 >> v1:172.21.15.60:6789/0 conn(0x7f9a28071e40 legacy=0x7f9a28096800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:52.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.274+0000 7f9a1e7fc640 1 -- 172.21.15.60:0/1887603518 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2809a280 con 0x7f9a28070110 2023-08-24T10:09:52.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.275+0000 7f9a1e7fc640 1 --2- 172.21.15.60:0/1887603518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a28070110 0x7f9a281356a0 crc :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e43c4114d06def server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:52.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.275+0000 7f9a267fc640 1 -- 172.21.15.60:0/1887603518 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9a10009dd0 con 0x7f9a28070110 2023-08-24T10:09:52.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.275+0000 7f9a267fc640 1 -- 172.21.15.60:0/1887603518 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f9a1000e230 con 0x7f9a28070110 2023-08-24T10:09:52.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.275+0000 7f9a267fc640 1 -- 172.21.15.60:0/1887603518 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9a1000e4d0 con 0x7f9a28070110 2023-08-24T10:09:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.275+0000 7f9a2d99e640 1 -- 172.21.15.60:0/1887603518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a28070110 msgr2=0x7f9a281356a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.275+0000 7f9a2d99e640 1 --2- 172.21.15.60:0/1887603518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a28070110 0x7f9a281356a0 crc :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:52.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.275+0000 7f9a2d99e640 1 -- 172.21.15.60:0/1887603518 shutdown_connections 2023-08-24T10:09:52.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.275+0000 7f9a2d99e640 1 --2- 172.21.15.60:0/1887603518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a28070110 0x7f9a281356a0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:52.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.275+0000 7f9a2d99e640 1 -- 172.21.15.60:0/1887603518 >> 172.21.15.60:0/1887603518 conn(0x7f9a2809e110 msgr2=0x7f9a2809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:52.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.275+0000 7f9a2d99e640 1 -- 172.21.15.60:0/1887603518 shutdown_connections 2023-08-24T10:09:52.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.275+0000 7f9a2d99e640 1 -- 172.21.15.60:0/1887603518 wait complete. 2023-08-24T10:09:52.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.276+0000 7f9a2d99e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:52.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.277+0000 7f9a2d99e640 1 Processor -- start 2023-08-24T10:09:52.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.277+0000 7f9a2d99e640 1 -- start start 2023-08-24T10:09:52.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.278+0000 7f9a2d99e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a28070110 0x7f9a2806cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:52.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.278+0000 7f9a2d99e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2806e280 con 0x7f9a28070110 2023-08-24T10:09:52.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.278+0000 7f9a26ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a28070110 0x7f9a2806cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:52.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.278+0000 7f9a26ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a28070110 0x7f9a2806cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36698/0 (socket says 172.21.15.60:36698) 2023-08-24T10:09:52.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.278+0000 7f9a26ffd640 1 -- 172.21.15.60:0/1178576313 learned_addr learned my addr 172.21.15.60:0/1178576313 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:52.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.278+0000 7f9a26ffd640 1 -- 172.21.15.60:0/1178576313 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2806e8f0 con 0x7f9a28070110 2023-08-24T10:09:52.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.278+0000 7f9a26ffd640 1 --2- 172.21.15.60:0/1178576313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a28070110 0x7f9a2806cef0 crc :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:52.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.278+0000 7f9a24ff9640 1 -- 172.21.15.60:0/1178576313 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9a1800c860 con 0x7f9a28070110 2023-08-24T10:09:52.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.278+0000 7f9a24ff9640 1 -- 172.21.15.60:0/1178576313 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f9a1800c9c0 con 0x7f9a28070110 2023-08-24T10:09:52.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.278+0000 7f9a24ff9640 1 -- 172.21.15.60:0/1178576313 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9a1800cc60 con 0x7f9a28070110 2023-08-24T10:09:52.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.278+0000 7f9a2d99e640 1 -- 172.21.15.60:0/1178576313 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a28125eb0 con 0x7f9a28070110 2023-08-24T10:09:52.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.278+0000 7f9a2d99e640 1 -- 172.21.15.60:0/1178576313 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2806ee20 con 0x7f9a28070110 2023-08-24T10:09:52.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.279+0000 7f9a24ff9640 1 -- 172.21.15.60:0/1178576313 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f9a18011310 con 0x7f9a28070110 2023-08-24T10:09:52.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.279+0000 7f9a24ff9640 1 --2- 172.21.15.60:0/1178576313 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9a08037180 0x7f9a08039690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:52.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.279+0000 7f9a24ff9640 1 -- 172.21.15.60:0/1178576313 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f9a18046a30 con 0x7f9a28070110 2023-08-24T10:09:52.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.279+0000 7f9a2d99e640 1 -- 172.21.15.60:0/1178576313 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a28097a50 con 0x7f9a28070110 2023-08-24T10:09:52.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.280+0000 7f9a1e7fc640 1 --2- 172.21.15.60:0/1178576313 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9a08037180 0x7f9a08039690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:52.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.280+0000 7f9a1e7fc640 1 --2- 172.21.15.60:0/1178576313 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9a08037180 0x7f9a08039690 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:52.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.284+0000 7f9a24ff9640 1 -- 172.21.15.60:0/1178576313 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f9a180102a0 con 0x7f9a28070110 2023-08-24T10:09:52.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.457+0000 7f9a2d99e640 1 -- 172.21.15.60:0/1178576313 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a28070970 con 0x7f9a08037180 2023-08-24T10:09:52.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.458+0000 7f9a24ff9640 1 -- 172.21.15.60:0/1178576313 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f9a10009da0 con 0x7f9a08037180 2023-08-24T10:09:52.461 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:52.461 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:52.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.460+0000 7f9a2d99e640 1 -- 172.21.15.60:0/1178576313 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9a08037180 msgr2=0x7f9a08039690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:52.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.460+0000 7f9a2d99e640 1 --2- 172.21.15.60:0/1178576313 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9a08037180 0x7f9a08039690 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:52.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.460+0000 7f9a2d99e640 1 -- 172.21.15.60:0/1178576313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a28070110 msgr2=0x7f9a2806cef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:52.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.460+0000 7f9a2d99e640 1 --2- 172.21.15.60:0/1178576313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a28070110 0x7f9a2806cef0 crc :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:52.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.460+0000 7f9a2d99e640 1 -- 172.21.15.60:0/1178576313 shutdown_connections 2023-08-24T10:09:52.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.460+0000 7f9a2d99e640 1 --2- 172.21.15.60:0/1178576313 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f9a08037180 0x7f9a08039690 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:52.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.460+0000 7f9a2d99e640 1 --2- 172.21.15.60:0/1178576313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a28070110 0x7f9a2806cef0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:52.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.460+0000 7f9a2d99e640 1 -- 172.21.15.60:0/1178576313 >> 172.21.15.60:0/1178576313 conn(0x7f9a2809e110 msgr2=0x7f9a28098490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:52.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.460+0000 7f9a2d99e640 1 -- 172.21.15.60:0/1178576313 shutdown_connections 2023-08-24T10:09:52.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.460+0000 7f9a2d99e640 1 -- 172.21.15.60:0/1178576313 wait complete. 2023-08-24T10:09:52.473 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2023-08-24T10:09:51.551837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:52.474 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:52.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.579+0000 7f71a64ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:52.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.583+0000 7f71a64ac640 1 Processor -- start 2023-08-24T10:09:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.583+0000 7f71a64ac640 1 -- start start 2023-08-24T10:09:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.584+0000 7f71a64ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f71a0131d40 0x7f71a0134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.584+0000 7f71a64ac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a0099440 con 0x7f71a006d300 2023-08-24T10:09:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.584+0000 7f71a64ac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a009a050 con 0x7f71a0131d40 2023-08-24T10:09:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.584+0000 7f719f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f71a0131d40 0x7f71a0134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.584+0000 7f719f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f71a0131d40 0x7f71a0134160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36700/0 (socket says 172.21.15.60:36700) 2023-08-24T10:09:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.584+0000 7f719ffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f71a006d300 0x7f71a012bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48624/0 (socket says 172.21.15.60:48624) 2023-08-24T10:09:52.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.584+0000 7f719f7fe640 1 -- 172.21.15.60:0/1491019338 learned_addr learned my addr 172.21.15.60:0/1491019338 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:52.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.584+0000 7f719f7fe640 1 -- 172.21.15.60:0/1491019338 >> v1:172.21.15.60:6789/0 conn(0x7f71a006d300 legacy=0x7f71a012bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:52.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.584+0000 7f719f7fe640 1 -- 172.21.15.60:0/1491019338 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a012b9d0 con 0x7f71a0131d40 2023-08-24T10:09:52.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.584+0000 7f719f7fe640 1 --2- 172.21.15.60:0/1491019338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71a0131d40 0x7f71a0134160 crc :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775b159c1c64d5c4 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:52.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.584+0000 7f719effd640 1 -- 172.21.15.60:0/1491019338 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7190009dc0 con 0x7f71a0131d40 2023-08-24T10:09:52.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.584+0000 7f719effd640 1 -- 172.21.15.60:0/1491019338 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f719000e230 con 0x7f71a0131d40 2023-08-24T10:09:52.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.584+0000 7f719effd640 1 -- 172.21.15.60:0/1491019338 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f719000e4d0 con 0x7f71a0131d40 2023-08-24T10:09:52.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.585+0000 7f71a64ac640 1 -- 172.21.15.60:0/1491019338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71a0131d40 msgr2=0x7f71a0134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:52.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.585+0000 7f71a64ac640 1 --2- 172.21.15.60:0/1491019338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71a0131d40 0x7f71a0134160 crc :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:52.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.585+0000 7f71a64ac640 1 -- 172.21.15.60:0/1491019338 shutdown_connections 2023-08-24T10:09:52.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.585+0000 7f71a64ac640 1 --2- 172.21.15.60:0/1491019338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71a0131d40 0x7f71a0134160 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.585+0000 7f71a64ac640 1 -- 172.21.15.60:0/1491019338 >> 172.21.15.60:0/1491019338 conn(0x7f71a009e110 msgr2=0x7f71a009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.585+0000 7f71a64ac640 1 -- 172.21.15.60:0/1491019338 shutdown_connections 2023-08-24T10:09:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.585+0000 7f71a64ac640 1 -- 172.21.15.60:0/1491019338 wait complete. 2023-08-24T10:09:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.585+0000 7f71a64ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:52.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.586+0000 7f71a64ac640 1 Processor -- start 2023-08-24T10:09:52.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.587+0000 7f71a64ac640 1 -- start start 2023-08-24T10:09:52.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.587+0000 7f71a64ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71a006d300 0x7f71a01c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:52.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.587+0000 7f71a64ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a0127640 con 0x7f71a006d300 2023-08-24T10:09:52.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.587+0000 7f719ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71a006d300 0x7f71a01c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:52.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.587+0000 7f719ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71a006d300 0x7f71a01c2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36704/0 (socket says 172.21.15.60:36704) 2023-08-24T10:09:52.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.587+0000 7f719ffff640 1 -- 172.21.15.60:0/116266930 learned_addr learned my addr 172.21.15.60:0/116266930 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:52.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.588+0000 7f719ffff640 1 -- 172.21.15.60:0/116266930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a006ffd0 con 0x7f71a006d300 2023-08-24T10:09:52.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.588+0000 7f719ffff640 1 --2- 172.21.15.60:0/116266930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71a006d300 0x7f71a01c2f20 crc :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:52.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.588+0000 7f719cff9640 1 -- 172.21.15.60:0/116266930 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f71880071e0 con 0x7f71a006d300 2023-08-24T10:09:52.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.588+0000 7f719cff9640 1 -- 172.21.15.60:0/116266930 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7188007340 con 0x7f71a006d300 2023-08-24T10:09:52.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.588+0000 7f71a64ac640 1 -- 172.21.15.60:0/116266930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a0071980 con 0x7f71a006d300 2023-08-24T10:09:52.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.588+0000 7f719cff9640 1 -- 172.21.15.60:0/116266930 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f71880075e0 con 0x7f71a006d300 2023-08-24T10:09:52.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.588+0000 7f71a64ac640 1 -- 172.21.15.60:0/116266930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a006f330 con 0x7f71a006d300 2023-08-24T10:09:52.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.589+0000 7f719cff9640 1 -- 172.21.15.60:0/116266930 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f71880057e0 con 0x7f71a006d300 2023-08-24T10:09:52.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.589+0000 7f71a64ac640 1 -- 172.21.15.60:0/116266930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a0061800 con 0x7f71a006d300 2023-08-24T10:09:52.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.589+0000 7f719cff9640 1 --2- 172.21.15.60:0/116266930 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7180037130 0x7f7180039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:52.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.589+0000 7f719cff9640 1 -- 172.21.15.60:0/116266930 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f71880462e0 con 0x7f71a006d300 2023-08-24T10:09:52.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.589+0000 7f719f7fe640 1 --2- 172.21.15.60:0/116266930 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7180037130 0x7f7180039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:52.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.590+0000 7f719f7fe640 1 --2- 172.21.15.60:0/116266930 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7180037130 0x7f7180039640 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:52.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.594+0000 7f719cff9640 1 -- 172.21.15.60:0/116266930 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f718800d330 con 0x7f71a006d300 2023-08-24T10:09:52.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.767+0000 7f71a64ac640 1 -- 172.21.15.60:0/116266930 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f71a006da20 con 0x7f7180037130 2023-08-24T10:09:52.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.768+0000 7f719cff9640 1 -- 172.21.15.60:0/116266930 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f7190009d90 con 0x7f7180037130 2023-08-24T10:09:52.770 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:52.771 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:52.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.770+0000 7f71a64ac640 1 -- 172.21.15.60:0/116266930 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7180037130 msgr2=0x7f7180039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:52.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.770+0000 7f71a64ac640 1 --2- 172.21.15.60:0/116266930 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7180037130 0x7f7180039640 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:52.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.770+0000 7f71a64ac640 1 -- 172.21.15.60:0/116266930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71a006d300 msgr2=0x7f71a01c2f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:52.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.770+0000 7f71a64ac640 1 --2- 172.21.15.60:0/116266930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71a006d300 0x7f71a01c2f20 crc :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:52.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.770+0000 7f71a64ac640 1 -- 172.21.15.60:0/116266930 shutdown_connections 2023-08-24T10:09:52.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.770+0000 7f71a64ac640 1 --2- 172.21.15.60:0/116266930 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f7180037130 0x7f7180039640 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:52.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.770+0000 7f71a64ac640 1 --2- 172.21.15.60:0/116266930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71a006d300 0x7f71a01c2f20 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:52.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.770+0000 7f71a64ac640 1 -- 172.21.15.60:0/116266930 >> 172.21.15.60:0/116266930 conn(0x7f71a009e110 msgr2=0x7f71a012ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:52.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.770+0000 7f71a64ac640 1 -- 172.21.15.60:0/116266930 shutdown_connections 2023-08-24T10:09:52.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.770+0000 7f71a64ac640 1 -- 172.21.15.60:0/116266930 wait complete. 2023-08-24T10:09:52.783 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2023-08-24T10:09:51.551837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:52.784 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:09:52.784 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:52.785 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:09:52.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:52.785 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:09:52.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:52.786 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:09:52.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:52.786 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:09:52.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:52.787 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:09:52.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:52.787 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:09:52.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:52.788 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:09:52.788 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:52.788 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:09:52.788 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:52.789 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:09:52.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.894+0000 7ff12d0ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:52.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.897+0000 7ff12d0ac640 1 Processor -- start 2023-08-24T10:09:52.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.897+0000 7ff12d0ac640 1 -- start start 2023-08-24T10:09:52.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.898+0000 7ff12d0ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff12812ece0 0x7ff12812f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:52.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.898+0000 7ff12d0ac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12809a650 con 0x7ff12812f630 2023-08-24T10:09:52.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.898+0000 7ff12d0ac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff12806da00 con 0x7ff12812ece0 2023-08-24T10:09:52.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.898+0000 7ff126d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff12812ece0 0x7ff12812f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:52.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.898+0000 7ff126d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff12812ece0 0x7ff12812f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36712/0 (socket says 172.21.15.60:36712) 2023-08-24T10:09:52.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.898+0000 7ff126d9d640 1 -- 172.21.15.60:0/2291047512 learned_addr learned my addr 172.21.15.60:0/2291047512 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:52.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.898+0000 7ff126d9d640 1 -- 172.21.15.60:0/2291047512 >> v1:172.21.15.60:6789/0 conn(0x7ff12812f630 legacy=0x7ff128097fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:52.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.898+0000 7ff126d9d640 1 -- 172.21.15.60:0/2291047512 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12812ea10 con 0x7ff12812ece0 2023-08-24T10:09:52.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.898+0000 7ff126d9d640 1 --2- 172.21.15.60:0/2291047512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff12812ece0 0x7ff12812f0f0 crc :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b363c51168b1b0c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:52.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.898+0000 7ff12659c640 1 -- 172.21.15.60:0/2291047512 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff118009dd0 con 0x7ff12812ece0 2023-08-24T10:09:52.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.898+0000 7ff12659c640 1 -- 172.21.15.60:0/2291047512 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ff11800e230 con 0x7ff12812ece0 2023-08-24T10:09:52.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.898+0000 7ff12659c640 1 -- 172.21.15.60:0/2291047512 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff11800e4d0 con 0x7ff12812ece0 2023-08-24T10:09:52.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.899+0000 7ff12d0ac640 1 -- 172.21.15.60:0/2291047512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff12812ece0 msgr2=0x7ff12812f0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:52.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.899+0000 7ff12d0ac640 1 --2- 172.21.15.60:0/2291047512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff12812ece0 0x7ff12812f0f0 crc :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:52.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.899+0000 7ff12d0ac640 1 -- 172.21.15.60:0/2291047512 shutdown_connections 2023-08-24T10:09:52.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.899+0000 7ff12d0ac640 1 --2- 172.21.15.60:0/2291047512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff12812ece0 0x7ff12812f0f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:52.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.899+0000 7ff12d0ac640 1 -- 172.21.15.60:0/2291047512 >> 172.21.15.60:0/2291047512 conn(0x7ff12809e110 msgr2=0x7ff12809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:52.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.899+0000 7ff12d0ac640 1 -- 172.21.15.60:0/2291047512 shutdown_connections 2023-08-24T10:09:52.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.899+0000 7ff12d0ac640 1 -- 172.21.15.60:0/2291047512 wait complete. 2023-08-24T10:09:52.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.899+0000 7ff12d0ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:52.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.900+0000 7ff12d0ac640 1 Processor -- start 2023-08-24T10:09:52.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.901+0000 7ff12d0ac640 1 -- start start 2023-08-24T10:09:52.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.901+0000 7ff12d0ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff12812ece0 0x7ff128133b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:52.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.901+0000 7ff12d0ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12806b200 con 0x7ff12812ece0 2023-08-24T10:09:52.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.901+0000 7ff126d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff12812ece0 0x7ff128133b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:52.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.901+0000 7ff126d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff12812ece0 0x7ff128133b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36714/0 (socket says 172.21.15.60:36714) 2023-08-24T10:09:52.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.901+0000 7ff126d9d640 1 -- 172.21.15.60:0/385799731 learned_addr learned my addr 172.21.15.60:0/385799731 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:52.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.902+0000 7ff126d9d640 1 -- 172.21.15.60:0/385799731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff128133840 con 0x7ff12812ece0 2023-08-24T10:09:52.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.902+0000 7ff126d9d640 1 --2- 172.21.15.60:0/385799731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff12812ece0 0x7ff128133b50 crc :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:52.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.902+0000 7ff11f7fe640 1 -- 172.21.15.60:0/385799731 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff118014410 con 0x7ff12812ece0 2023-08-24T10:09:52.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.902+0000 7ff11f7fe640 1 -- 172.21.15.60:0/385799731 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ff118014570 con 0x7ff12812ece0 2023-08-24T10:09:52.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.902+0000 7ff12d0ac640 1 -- 172.21.15.60:0/385799731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff128126990 con 0x7ff12812ece0 2023-08-24T10:09:52.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.902+0000 7ff11f7fe640 1 -- 172.21.15.60:0/385799731 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff118014810 con 0x7ff12812ece0 2023-08-24T10:09:52.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.902+0000 7ff12d0ac640 1 -- 172.21.15.60:0/385799731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff128125f20 con 0x7ff12812ece0 2023-08-24T10:09:52.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.903+0000 7ff11f7fe640 1 -- 172.21.15.60:0/385799731 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7ff1180149c0 con 0x7ff12812ece0 2023-08-24T10:09:52.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.903+0000 7ff12d0ac640 1 -- 172.21.15.60:0/385799731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff128134f10 con 0x7ff12812ece0 2023-08-24T10:09:52.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.903+0000 7ff11f7fe640 1 --2- 172.21.15.60:0/385799731 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff108037130 0x7ff108039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:52.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.903+0000 7ff11f7fe640 1 -- 172.21.15.60:0/385799731 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7ff118019040 con 0x7ff12812ece0 2023-08-24T10:09:52.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.903+0000 7ff11ffff640 1 --2- 172.21.15.60:0/385799731 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff108037130 0x7ff108039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:52.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.904+0000 7ff11ffff640 1 --2- 172.21.15.60:0/385799731 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff108037130 0x7ff108039640 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:52.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:52.908+0000 7ff11f7fe640 1 -- 172.21.15.60:0/385799731 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7ff11800f8f0 con 0x7ff12812ece0 2023-08-24T10:09:53.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.154+0000 7ff12d0ac640 1 -- 172.21.15.60:0/385799731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff128061800 con 0x7ff12812ece0 2023-08-24T10:09:53.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.154+0000 7ff11f7fe640 1 -- 172.21.15.60:0/385799731 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7ff118005960 con 0x7ff12812ece0 2023-08-24T10:09:53.157 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:53.157 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":303,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:09:53.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.156+0000 7ff12d0ac640 1 -- 172.21.15.60:0/385799731 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff108037130 msgr2=0x7ff108039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:53.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.156+0000 7ff12d0ac640 1 --2- 172.21.15.60:0/385799731 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff108037130 0x7ff108039640 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:53.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.156+0000 7ff12d0ac640 1 -- 172.21.15.60:0/385799731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff12812ece0 msgr2=0x7ff128133b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:53.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.156+0000 7ff12d0ac640 1 --2- 172.21.15.60:0/385799731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff12812ece0 0x7ff128133b50 crc :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:53.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.156+0000 7ff12d0ac640 1 -- 172.21.15.60:0/385799731 shutdown_connections 2023-08-24T10:09:53.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.156+0000 7ff12d0ac640 1 --2- 172.21.15.60:0/385799731 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7ff108037130 0x7ff108039640 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:53.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.156+0000 7ff12d0ac640 1 --2- 172.21.15.60:0/385799731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff12812ece0 0x7ff128133b50 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:53.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.156+0000 7ff12d0ac640 1 -- 172.21.15.60:0/385799731 >> 172.21.15.60:0/385799731 conn(0x7ff12809e110 msgr2=0x7ff128097710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:53.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.156+0000 7ff12d0ac640 1 -- 172.21.15.60:0/385799731 shutdown_connections 2023-08-24T10:09:53.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.156+0000 7ff12d0ac640 1 -- 172.21.15.60:0/385799731 wait complete. 2023-08-24T10:09:53.169 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:09:53.169 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:53.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.275+0000 7f5ed8b5e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:53.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.279+0000 7f5ed8b5e640 1 Processor -- start 2023-08-24T10:09:53.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.279+0000 7f5ed8b5e640 1 -- start start 2023-08-24T10:09:53.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.279+0000 7f5ed8b5e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ed4132b50 0x7f5ed4132f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:53.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.279+0000 7f5ed8b5e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed4132610 con 0x7f5ed412f110 2023-08-24T10:09:53.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.279+0000 7f5ed8b5e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed409c260 con 0x7f5ed4132b50 2023-08-24T10:09:53.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.279+0000 7f5ed259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ed4132b50 0x7f5ed4132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:53.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.280+0000 7f5ed259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ed4132b50 0x7f5ed4132f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36720/0 (socket says 172.21.15.60:36720) 2023-08-24T10:09:53.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.280+0000 7f5ed259c640 1 -- 172.21.15.60:0/3409748401 learned_addr learned my addr 172.21.15.60:0/3409748401 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:53.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.280+0000 7f5ecbfff640 1 --1- 172.21.15.60:0/3409748401 >> v1:172.21.15.60:6789/0 conn(0x7f5ed412f110 0x7f5ed412f520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48656/0 (socket says 172.21.15.60:48656) 2023-08-24T10:09:53.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.280+0000 7f5ed259c640 1 -- 172.21.15.60:0/3409748401 >> v1:172.21.15.60:6789/0 conn(0x7f5ed412f110 legacy=0x7f5ed412f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:53.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.280+0000 7f5ed259c640 1 -- 172.21.15.60:0/3409748401 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed4132840 con 0x7f5ed4132b50 2023-08-24T10:09:53.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.280+0000 7f5ed259c640 1 --2- 172.21.15.60:0/3409748401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ed4132b50 0x7f5ed4132f60 crc :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9dd14203b37f345 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:53.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.280+0000 7f5ed1d9b640 1 -- 172.21.15.60:0/3409748401 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5ec4009dd0 con 0x7f5ed4132b50 2023-08-24T10:09:53.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.280+0000 7f5ed1d9b640 1 -- 172.21.15.60:0/3409748401 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5ec400e230 con 0x7f5ed4132b50 2023-08-24T10:09:53.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.280+0000 7f5ed1d9b640 1 -- 172.21.15.60:0/3409748401 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5ec400e4d0 con 0x7f5ed4132b50 2023-08-24T10:09:53.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.281+0000 7f5ed8b5e640 1 -- 172.21.15.60:0/3409748401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ed4132b50 msgr2=0x7f5ed4132f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:53.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.281+0000 7f5ed8b5e640 1 --2- 172.21.15.60:0/3409748401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ed4132b50 0x7f5ed4132f60 crc :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:53.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.281+0000 7f5ed8b5e640 1 -- 172.21.15.60:0/3409748401 shutdown_connections 2023-08-24T10:09:53.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.281+0000 7f5ed8b5e640 1 --2- 172.21.15.60:0/3409748401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ed4132b50 0x7f5ed4132f60 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:53.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.281+0000 7f5ed8b5e640 1 -- 172.21.15.60:0/3409748401 >> 172.21.15.60:0/3409748401 conn(0x7f5ed40a26a0 msgr2=0x7f5ed40a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:53.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.281+0000 7f5ed8b5e640 1 -- 172.21.15.60:0/3409748401 shutdown_connections 2023-08-24T10:09:53.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.281+0000 7f5ed8b5e640 1 -- 172.21.15.60:0/3409748401 wait complete. 2023-08-24T10:09:53.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.281+0000 7f5ed8b5e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:53.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.282+0000 7f5ed8b5e640 1 Processor -- start 2023-08-24T10:09:53.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.282+0000 7f5ed8b5e640 1 -- start start 2023-08-24T10:09:53.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.283+0000 7f5ed8b5e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ed412f110 0x7f5ed409fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:53.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.283+0000 7f5ed8b5e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed412cc80 con 0x7f5ed412f110 2023-08-24T10:09:53.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.283+0000 7f5ed259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ed412f110 0x7f5ed409fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:53.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.283+0000 7f5ed259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ed412f110 0x7f5ed409fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36726/0 (socket says 172.21.15.60:36726) 2023-08-24T10:09:53.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.283+0000 7f5ed259c640 1 -- 172.21.15.60:0/2258979286 learned_addr learned my addr 172.21.15.60:0/2258979286 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:53.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.283+0000 7f5ed259c640 1 -- 172.21.15.60:0/2258979286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed409f400 con 0x7f5ed412f110 2023-08-24T10:09:53.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.283+0000 7f5ed259c640 1 --2- 172.21.15.60:0/2258979286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ed412f110 0x7f5ed409fcd0 crc :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:53.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.283+0000 7f5ecaffd640 1 -- 172.21.15.60:0/2258979286 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5ec4014540 con 0x7f5ed412f110 2023-08-24T10:09:53.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.283+0000 7f5ecaffd640 1 -- 172.21.15.60:0/2258979286 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5ec40146a0 con 0x7f5ed412f110 2023-08-24T10:09:53.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.283+0000 7f5ecaffd640 1 -- 172.21.15.60:0/2258979286 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5ec4014940 con 0x7f5ed412f110 2023-08-24T10:09:53.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.283+0000 7f5ed8b5e640 1 -- 172.21.15.60:0/2258979286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed40a18e0 con 0x7f5ed412f110 2023-08-24T10:09:53.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.284+0000 7f5ed8b5e640 1 -- 172.21.15.60:0/2258979286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed406a1d0 con 0x7f5ed412f110 2023-08-24T10:09:53.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.284+0000 7f5ecaffd640 1 -- 172.21.15.60:0/2258979286 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f5ec4014af0 con 0x7f5ed412f110 2023-08-24T10:09:53.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.284+0000 7f5ecaffd640 1 --2- 172.21.15.60:0/2258979286 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5eb4037130 0x7f5eb4039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:53.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.284+0000 7f5ed8b5e640 1 -- 172.21.15.60:0/2258979286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed411c2b0 con 0x7f5ed412f110 2023-08-24T10:09:53.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.285+0000 7f5ecaffd640 1 -- 172.21.15.60:0/2258979286 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f5ec4019040 con 0x7f5ed412f110 2023-08-24T10:09:53.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.285+0000 7f5ecbfff640 1 --2- 172.21.15.60:0/2258979286 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5eb4037130 0x7f5eb4039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:53.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.285+0000 7f5ecbfff640 1 --2- 172.21.15.60:0/2258979286 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5eb4037130 0x7f5eb4039640 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:53.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.289+0000 7f5ecaffd640 1 -- 172.21.15.60:0/2258979286 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f5ec4014df0 con 0x7f5ed412f110 2023-08-24T10:09:53.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.464+0000 7f5ed8b5e640 1 -- 172.21.15.60:0/2258979286 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ed41334a0 con 0x7f5eb4037130 2023-08-24T10:09:53.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.466+0000 7f5ecaffd640 1 -- 172.21.15.60:0/2258979286 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f5ebc0011a0 con 0x7f5eb4037130 2023-08-24T10:09:53.468 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:53.468 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:53.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.467+0000 7f5ed8b5e640 1 -- 172.21.15.60:0/2258979286 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5eb4037130 msgr2=0x7f5eb4039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.467+0000 7f5ed8b5e640 1 --2- 172.21.15.60:0/2258979286 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5eb4037130 0x7f5eb4039640 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.467+0000 7f5ed8b5e640 1 -- 172.21.15.60:0/2258979286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ed412f110 msgr2=0x7f5ed409fcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.467+0000 7f5ed8b5e640 1 --2- 172.21.15.60:0/2258979286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ed412f110 0x7f5ed409fcd0 crc :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.467+0000 7f5ed8b5e640 1 -- 172.21.15.60:0/2258979286 shutdown_connections 2023-08-24T10:09:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.467+0000 7f5ed8b5e640 1 --2- 172.21.15.60:0/2258979286 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5eb4037130 0x7f5eb4039640 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.467+0000 7f5ed8b5e640 1 --2- 172.21.15.60:0/2258979286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ed412f110 0x7f5ed409fcd0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:53.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.467+0000 7f5ed8b5e640 1 -- 172.21.15.60:0/2258979286 >> 172.21.15.60:0/2258979286 conn(0x7f5ed40a26a0 msgr2=0x7f5ed41ba920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:53.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.467+0000 7f5ed8b5e640 1 -- 172.21.15.60:0/2258979286 shutdown_connections 2023-08-24T10:09:53.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:53.467+0000 7f5ed8b5e640 1 -- 172.21.15.60:0/2258979286 wait complete. 2023-08-24T10:09:53.480 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2023-08-24T10:09:51.551837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:53.668 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~286s 2023-08-24T10:09:53.668 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~286s 2023-08-24T10:09:53.668 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~286s 2023-08-24T10:09:53.669 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~286s 2023-08-24T10:09:56.481 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:56.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.601+0000 7f074764e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:56.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.605+0000 7f074764e640 1 Processor -- start 2023-08-24T10:09:56.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.605+0000 7f074764e640 1 -- start start 2023-08-24T10:09:56.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.606+0000 7f074764e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0740132630 0x7f0740134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:56.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.606+0000 7f074764e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f074006e320 con 0x7f0740096860 2023-08-24T10:09:56.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.606+0000 7f074764e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f074006db70 con 0x7f0740132630 2023-08-24T10:09:56.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.606+0000 7f0744be9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0740132630 0x7f0740134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:56.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.606+0000 7f0744be9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0740132630 0x7f0740134a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54220/0 (socket says 172.21.15.60:54220) 2023-08-24T10:09:56.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.606+0000 7f0744be9640 1 -- 172.21.15.60:0/785872803 learned_addr learned my addr 172.21.15.60:0/785872803 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:56.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.607+0000 7f0744be9640 1 -- 172.21.15.60:0/785872803 >> v1:172.21.15.60:6789/0 conn(0x7f0740096860 legacy=0x7f0740096c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:56.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.607+0000 7f0744be9640 1 -- 172.21.15.60:0/785872803 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074006cef0 con 0x7f0740132630 2023-08-24T10:09:56.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.607+0000 7f0744be9640 1 --2- 172.21.15.60:0/785872803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0740132630 0x7f0740134a50 crc :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45781a9e0a57d987 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:56.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.607+0000 7f0737fff640 1 -- 172.21.15.60:0/785872803 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0730009dd0 con 0x7f0740132630 2023-08-24T10:09:56.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.607+0000 7f0737fff640 1 -- 172.21.15.60:0/785872803 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f073000e230 con 0x7f0740132630 2023-08-24T10:09:56.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.607+0000 7f0737fff640 1 -- 172.21.15.60:0/785872803 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f073000e4d0 con 0x7f0740132630 2023-08-24T10:09:56.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.608+0000 7f074764e640 1 -- 172.21.15.60:0/785872803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0740132630 msgr2=0x7f0740134a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:56.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.608+0000 7f074764e640 1 --2- 172.21.15.60:0/785872803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0740132630 0x7f0740134a50 crc :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:56.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.608+0000 7f074764e640 1 -- 172.21.15.60:0/785872803 shutdown_connections 2023-08-24T10:09:56.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.608+0000 7f074764e640 1 --2- 172.21.15.60:0/785872803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0740132630 0x7f0740134a50 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:56.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.608+0000 7f074764e640 1 -- 172.21.15.60:0/785872803 >> 172.21.15.60:0/785872803 conn(0x7f074009e110 msgr2=0x7f074009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:56.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.608+0000 7f074764e640 1 -- 172.21.15.60:0/785872803 shutdown_connections 2023-08-24T10:09:56.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.608+0000 7f074764e640 1 -- 172.21.15.60:0/785872803 wait complete. 2023-08-24T10:09:56.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.608+0000 7f074764e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:56.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.609+0000 7f074764e640 1 Processor -- start 2023-08-24T10:09:56.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.609+0000 7f074764e640 1 -- start start 2023-08-24T10:09:56.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.610+0000 7f074764e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0740096860 0x7f07401c1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:56.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.610+0000 7f074764e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074008e680 con 0x7f0740096860 2023-08-24T10:09:56.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.610+0000 7f07453ea640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0740096860 0x7f07401c1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:56.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.610+0000 7f07453ea640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0740096860 0x7f07401c1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54232/0 (socket says 172.21.15.60:54232) 2023-08-24T10:09:56.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.610+0000 7f07453ea640 1 -- 172.21.15.60:0/4255232286 learned_addr learned my addr 172.21.15.60:0/4255232286 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:56.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.610+0000 7f07453ea640 1 -- 172.21.15.60:0/4255232286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0740090c20 con 0x7f0740096860 2023-08-24T10:09:56.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.610+0000 7f07453ea640 1 --2- 172.21.15.60:0/4255232286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0740096860 0x7f07401c1d80 crc :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:56.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.610+0000 7f0735ffb640 1 -- 172.21.15.60:0/4255232286 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f073c00c7e0 con 0x7f0740096860 2023-08-24T10:09:56.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.610+0000 7f0735ffb640 1 -- 172.21.15.60:0/4255232286 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f073c00c940 con 0x7f0740096860 2023-08-24T10:09:56.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.610+0000 7f074764e640 1 -- 172.21.15.60:0/4255232286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07401c19e0 con 0x7f0740096860 2023-08-24T10:09:56.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.610+0000 7f0735ffb640 1 -- 172.21.15.60:0/4255232286 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f073c00cbe0 con 0x7f0740096860 2023-08-24T10:09:56.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.611+0000 7f074764e640 1 -- 172.21.15.60:0/4255232286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074008f720 con 0x7f0740096860 2023-08-24T10:09:56.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.611+0000 7f0735ffb640 1 -- 172.21.15.60:0/4255232286 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f073c011310 con 0x7f0740096860 2023-08-24T10:09:56.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.611+0000 7f0735ffb640 1 --2- 172.21.15.60:0/4255232286 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0724037180 0x7f0724039690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:56.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.612+0000 7f0735ffb640 1 -- 172.21.15.60:0/4255232286 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f073c046b40 con 0x7f0740096860 2023-08-24T10:09:56.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.612+0000 7f074764e640 1 -- 172.21.15.60:0/4255232286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0740061800 con 0x7f0740096860 2023-08-24T10:09:56.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.612+0000 7f0744be9640 1 --2- 172.21.15.60:0/4255232286 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0724037180 0x7f0724039690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:56.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.612+0000 7f0744be9640 1 --2- 172.21.15.60:0/4255232286 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0724037180 0x7f0724039690 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:56.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.616+0000 7f0735ffb640 1 -- 172.21.15.60:0/4255232286 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f073c00ad00 con 0x7f0740096860 2023-08-24T10:09:56.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.793+0000 7f074764e640 1 -- 172.21.15.60:0/4255232286 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f074006d190 con 0x7f0724037180 2023-08-24T10:09:56.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.795+0000 7f0735ffb640 1 -- 172.21.15.60:0/4255232286 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f0730009df0 con 0x7f0724037180 2023-08-24T10:09:56.797 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:56.797 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:56.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.796+0000 7f074764e640 1 -- 172.21.15.60:0/4255232286 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0724037180 msgr2=0x7f0724039690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:56.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.796+0000 7f074764e640 1 --2- 172.21.15.60:0/4255232286 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0724037180 0x7f0724039690 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:56.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.796+0000 7f074764e640 1 -- 172.21.15.60:0/4255232286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0740096860 msgr2=0x7f07401c1d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:56.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.796+0000 7f074764e640 1 --2- 172.21.15.60:0/4255232286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0740096860 0x7f07401c1d80 crc :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:56.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.796+0000 7f074764e640 1 -- 172.21.15.60:0/4255232286 shutdown_connections 2023-08-24T10:09:56.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.796+0000 7f074764e640 1 --2- 172.21.15.60:0/4255232286 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0724037180 0x7f0724039690 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:56.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.796+0000 7f074764e640 1 --2- 172.21.15.60:0/4255232286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0740096860 0x7f07401c1d80 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:56.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.796+0000 7f074764e640 1 -- 172.21.15.60:0/4255232286 >> 172.21.15.60:0/4255232286 conn(0x7f074009e110 msgr2=0x7f0740131180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:56.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.796+0000 7f074764e640 1 -- 172.21.15.60:0/4255232286 shutdown_connections 2023-08-24T10:09:56.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.796+0000 7f074764e640 1 -- 172.21.15.60:0/4255232286 wait complete. 2023-08-24T10:09:56.808 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2023-08-24T10:09:55.552423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:56.809 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:56.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.915+0000 7f09cb4ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.918+0000 7f09cb4ec640 1 Processor -- start 2023-08-24T10:09:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.919+0000 7f09cb4ec640 1 -- start start 2023-08-24T10:09:56.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.919+0000 7f09cb4ec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f09c4135980 0x7f09c4137da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:56.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.919+0000 7f09cb4ec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c4132610 con 0x7f09c4132b50 2023-08-24T10:09:56.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.919+0000 7f09cb4ec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c409c260 con 0x7f09c4135980 2023-08-24T10:09:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.919+0000 7f09c8a87640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f09c4135980 0x7f09c4137da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.919+0000 7f09c8a87640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f09c4135980 0x7f09c4137da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54242/0 (socket says 172.21.15.60:54242) 2023-08-24T10:09:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.919+0000 7f09c8a87640 1 -- 172.21.15.60:0/17505669 learned_addr learned my addr 172.21.15.60:0/17505669 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.920+0000 7f09c8a87640 1 -- 172.21.15.60:0/17505669 >> v1:172.21.15.60:6789/0 conn(0x7f09c4132b50 legacy=0x7f09c4132f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.920+0000 7f09c8a87640 1 -- 172.21.15.60:0/17505669 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c4132840 con 0x7f09c4135980 2023-08-24T10:09:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.920+0000 7f09c8a87640 1 --2- 172.21.15.60:0/17505669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c4135980 0x7f09c4137da0 crc :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2eb422e9ae1b1a8 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:56.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.920+0000 7f09bbfff640 1 -- 172.21.15.60:0/17505669 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f09b4009dd0 con 0x7f09c4135980 2023-08-24T10:09:56.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.920+0000 7f09bbfff640 1 -- 172.21.15.60:0/17505669 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f09b400e230 con 0x7f09c4135980 2023-08-24T10:09:56.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.920+0000 7f09bbfff640 1 -- 172.21.15.60:0/17505669 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f09b400e4d0 con 0x7f09c4135980 2023-08-24T10:09:56.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.921+0000 7f09cb4ec640 1 -- 172.21.15.60:0/17505669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c4135980 msgr2=0x7f09c4137da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:56.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.921+0000 7f09cb4ec640 1 --2- 172.21.15.60:0/17505669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c4135980 0x7f09c4137da0 crc :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:56.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.921+0000 7f09cb4ec640 1 -- 172.21.15.60:0/17505669 shutdown_connections 2023-08-24T10:09:56.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.921+0000 7f09cb4ec640 1 --2- 172.21.15.60:0/17505669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c4135980 0x7f09c4137da0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:56.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.921+0000 7f09cb4ec640 1 -- 172.21.15.60:0/17505669 >> 172.21.15.60:0/17505669 conn(0x7f09c40a26a0 msgr2=0x7f09c40a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:56.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.921+0000 7f09cb4ec640 1 -- 172.21.15.60:0/17505669 shutdown_connections 2023-08-24T10:09:56.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.921+0000 7f09cb4ec640 1 -- 172.21.15.60:0/17505669 wait complete. 2023-08-24T10:09:56.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.921+0000 7f09cb4ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:56.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.922+0000 7f09cb4ec640 1 Processor -- start 2023-08-24T10:09:56.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.922+0000 7f09cb4ec640 1 -- start start 2023-08-24T10:09:56.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.923+0000 7f09cb4ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c4132b50 0x7f09c40a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:56.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.923+0000 7f09cb4ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c40a1b60 con 0x7f09c4132b50 2023-08-24T10:09:56.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.923+0000 7f09c9288640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c4132b50 0x7f09c40a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:56.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.923+0000 7f09c9288640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c4132b50 0x7f09c40a0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54244/0 (socket says 172.21.15.60:54244) 2023-08-24T10:09:56.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.923+0000 7f09c9288640 1 -- 172.21.15.60:0/3965645046 learned_addr learned my addr 172.21.15.60:0/3965645046 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:56.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.923+0000 7f09c9288640 1 -- 172.21.15.60:0/3965645046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c409fab0 con 0x7f09c4132b50 2023-08-24T10:09:56.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.923+0000 7f09c9288640 1 --2- 172.21.15.60:0/3965645046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c4132b50 0x7f09c40a0000 crc :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:56.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.923+0000 7f09b9ffb640 1 -- 172.21.15.60:0/3965645046 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f09c000c860 con 0x7f09c4132b50 2023-08-24T10:09:56.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.923+0000 7f09b9ffb640 1 -- 172.21.15.60:0/3965645046 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f09c000c9c0 con 0x7f09c4132b50 2023-08-24T10:09:56.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.923+0000 7f09cb4ec640 1 -- 172.21.15.60:0/3965645046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c406a240 con 0x7f09c4132b50 2023-08-24T10:09:56.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.923+0000 7f09b9ffb640 1 -- 172.21.15.60:0/3965645046 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f09c000cc60 con 0x7f09c4132b50 2023-08-24T10:09:56.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.923+0000 7f09cb4ec640 1 -- 172.21.15.60:0/3965645046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c411ac50 con 0x7f09c4132b50 2023-08-24T10:09:56.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.924+0000 7f09b9ffb640 1 -- 172.21.15.60:0/3965645046 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f09c0011310 con 0x7f09c4132b50 2023-08-24T10:09:56.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.924+0000 7f09b9ffb640 1 --2- 172.21.15.60:0/3965645046 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f09a8037130 0x7f09a8039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:56.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.924+0000 7f09b9ffb640 1 -- 172.21.15.60:0/3965645046 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f09c0046850 con 0x7f09c4132b50 2023-08-24T10:09:56.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.924+0000 7f09cb4ec640 1 -- 172.21.15.60:0/3965645046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c406a240 con 0x7f09c4132b50 2023-08-24T10:09:56.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.925+0000 7f09c8a87640 1 --2- 172.21.15.60:0/3965645046 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f09a8037130 0x7f09a8039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:56.932 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.925+0000 7f09c8a87640 1 --2- 172.21.15.60:0/3965645046 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f09a8037130 0x7f09a8039640 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:56.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:56.929+0000 7f09b9ffb640 1 -- 172.21.15.60:0/3965645046 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f09c406a240 con 0x7f09c4132b50 2023-08-24T10:09:57.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.099+0000 7f09cb4ec640 1 -- 172.21.15.60:0/3965645046 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09c409f940 con 0x7f09a8037130 2023-08-24T10:09:57.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.101+0000 7f09b9ffb640 1 -- 172.21.15.60:0/3965645046 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f09c409f940 con 0x7f09a8037130 2023-08-24T10:09:57.103 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:57.104 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.103+0000 7f09cb4ec640 1 -- 172.21.15.60:0/3965645046 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f09a8037130 msgr2=0x7f09a8039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.103+0000 7f09cb4ec640 1 --2- 172.21.15.60:0/3965645046 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f09a8037130 0x7f09a8039640 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.103+0000 7f09cb4ec640 1 -- 172.21.15.60:0/3965645046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c4132b50 msgr2=0x7f09c40a0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.103+0000 7f09cb4ec640 1 --2- 172.21.15.60:0/3965645046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c4132b50 0x7f09c40a0000 crc :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.103+0000 7f09cb4ec640 1 -- 172.21.15.60:0/3965645046 shutdown_connections 2023-08-24T10:09:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.103+0000 7f09cb4ec640 1 --2- 172.21.15.60:0/3965645046 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f09a8037130 0x7f09a8039640 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:57.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.103+0000 7f09cb4ec640 1 --2- 172.21.15.60:0/3965645046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c4132b50 0x7f09c40a0000 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:57.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.103+0000 7f09cb4ec640 1 -- 172.21.15.60:0/3965645046 >> 172.21.15.60:0/3965645046 conn(0x7f09c40a26a0 msgr2=0x7f09c41baaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:57.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.103+0000 7f09cb4ec640 1 -- 172.21.15.60:0/3965645046 shutdown_connections 2023-08-24T10:09:57.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.103+0000 7f09cb4ec640 1 -- 172.21.15.60:0/3965645046 wait complete. 2023-08-24T10:09:57.115 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2023-08-24T10:09:55.552423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:57.117 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:09:57.118 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:57.118 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:09:57.119 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:57.119 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:09:57.119 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:57.119 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:09:57.120 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:57.120 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:09:57.120 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:57.121 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:09:57.121 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:57.121 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:09:57.121 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:57.121 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:09:57.122 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:57.122 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:09:57.122 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:09:57.123 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:09:57.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.228+0000 7f1fbb0ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:57.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.231+0000 7f1fbb0ac640 1 Processor -- start 2023-08-24T10:09:57.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.232+0000 7f1fbb0ac640 1 -- start start 2023-08-24T10:09:57.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.232+0000 7f1fbb0ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1fb412ece0 0x7f1fb412f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:57.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.232+0000 7f1fbb0ac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb409a650 con 0x7f1fb412f630 2023-08-24T10:09:57.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.232+0000 7f1fbb0ac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb406da00 con 0x7f1fb412ece0 2023-08-24T10:09:57.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.232+0000 7f1fb8e48640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1fb412ece0 0x7f1fb412f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:57.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.232+0000 7f1fb8e48640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1fb412ece0 0x7f1fb412f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54246/0 (socket says 172.21.15.60:54246) 2023-08-24T10:09:57.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.232+0000 7f1fb8e48640 1 -- 172.21.15.60:0/2688034207 learned_addr learned my addr 172.21.15.60:0/2688034207 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:57.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.233+0000 7f1fb8e48640 1 -- 172.21.15.60:0/2688034207 >> v1:172.21.15.60:6789/0 conn(0x7f1fb412f630 legacy=0x7f1fb4097fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:57.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.233+0000 7f1fb8e48640 1 -- 172.21.15.60:0/2688034207 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb412ea10 con 0x7f1fb412ece0 2023-08-24T10:09:57.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.233+0000 7f1fb8e48640 1 --2- 172.21.15.60:0/2688034207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb412ece0 0x7f1fb412f0f0 crc :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6652d81d788b423b server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:57.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.233+0000 7f1fb37fe640 1 -- 172.21.15.60:0/2688034207 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1fa4009dd0 con 0x7f1fb412ece0 2023-08-24T10:09:57.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.233+0000 7f1fb37fe640 1 -- 172.21.15.60:0/2688034207 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1fa400e230 con 0x7f1fb412ece0 2023-08-24T10:09:57.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.233+0000 7f1fb37fe640 1 -- 172.21.15.60:0/2688034207 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1fa400e4d0 con 0x7f1fb412ece0 2023-08-24T10:09:57.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.234+0000 7f1fbb0ac640 1 -- 172.21.15.60:0/2688034207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb412ece0 msgr2=0x7f1fb412f0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:57.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.234+0000 7f1fbb0ac640 1 --2- 172.21.15.60:0/2688034207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb412ece0 0x7f1fb412f0f0 crc :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:57.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.234+0000 7f1fbb0ac640 1 -- 172.21.15.60:0/2688034207 shutdown_connections 2023-08-24T10:09:57.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.234+0000 7f1fbb0ac640 1 --2- 172.21.15.60:0/2688034207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb412ece0 0x7f1fb412f0f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:57.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.234+0000 7f1fbb0ac640 1 -- 172.21.15.60:0/2688034207 >> 172.21.15.60:0/2688034207 conn(0x7f1fb409e110 msgr2=0x7f1fb409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:57.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.234+0000 7f1fbb0ac640 1 -- 172.21.15.60:0/2688034207 shutdown_connections 2023-08-24T10:09:57.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.234+0000 7f1fbb0ac640 1 -- 172.21.15.60:0/2688034207 wait complete. 2023-08-24T10:09:57.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.234+0000 7f1fbb0ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:57.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.235+0000 7f1fbb0ac640 1 Processor -- start 2023-08-24T10:09:57.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.235+0000 7f1fbb0ac640 1 -- start start 2023-08-24T10:09:57.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.236+0000 7f1fbb0ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb412ece0 0x7f1fb4129410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:57.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.236+0000 7f1fbb0ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb41288f0 con 0x7f1fb412ece0 2023-08-24T10:09:57.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.236+0000 7f1fb8e48640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb412ece0 0x7f1fb4129410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:57.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.236+0000 7f1fb8e48640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb412ece0 0x7f1fb4129410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54254/0 (socket says 172.21.15.60:54254) 2023-08-24T10:09:57.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.236+0000 7f1fb8e48640 1 -- 172.21.15.60:0/299776215 learned_addr learned my addr 172.21.15.60:0/299776215 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:57.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.236+0000 7f1fb8e48640 1 -- 172.21.15.60:0/299776215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb4128ec0 con 0x7f1fb412ece0 2023-08-24T10:09:57.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.236+0000 7f1fb8e48640 1 --2- 172.21.15.60:0/299776215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb412ece0 0x7f1fb4129410 crc :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:57.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.237+0000 7f1fb17fa640 1 -- 172.21.15.60:0/299776215 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1fa4014410 con 0x7f1fb412ece0 2023-08-24T10:09:57.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.237+0000 7f1fb17fa640 1 -- 172.21.15.60:0/299776215 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1fa4014570 con 0x7f1fb412ece0 2023-08-24T10:09:57.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.237+0000 7f1fbb0ac640 1 -- 172.21.15.60:0/299776215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb4089610 con 0x7f1fb412ece0 2023-08-24T10:09:57.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.237+0000 7f1fb17fa640 1 -- 172.21.15.60:0/299776215 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1fa4014810 con 0x7f1fb412ece0 2023-08-24T10:09:57.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.237+0000 7f1fbb0ac640 1 -- 172.21.15.60:0/299776215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb4128b20 con 0x7f1fb412ece0 2023-08-24T10:09:57.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.237+0000 7f1fb17fa640 1 -- 172.21.15.60:0/299776215 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f1fa40149c0 con 0x7f1fb412ece0 2023-08-24T10:09:57.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.238+0000 7f1fbb0ac640 1 -- 172.21.15.60:0/299776215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb4061800 con 0x7f1fb412ece0 2023-08-24T10:09:57.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.238+0000 7f1fb17fa640 1 --2- 172.21.15.60:0/299776215 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1fa0037130 0x7f1fa0039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:57.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.238+0000 7f1fb17fa640 1 -- 172.21.15.60:0/299776215 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f1fa4019040 con 0x7f1fb412ece0 2023-08-24T10:09:57.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.238+0000 7f1fb3fff640 1 --2- 172.21.15.60:0/299776215 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1fa0037130 0x7f1fa0039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:57.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.238+0000 7f1fb3fff640 1 --2- 172.21.15.60:0/299776215 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1fa0037130 0x7f1fa0039640 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:57.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.242+0000 7f1fb17fa640 1 -- 172.21.15.60:0/299776215 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f1fa400f8f0 con 0x7f1fb412ece0 2023-08-24T10:09:57.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.485+0000 7f1fbb0ac640 1 -- 172.21.15.60:0/299776215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1fb4129d70 con 0x7f1fb412ece0 2023-08-24T10:09:57.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.485+0000 7f1fb17fa640 1 -- 172.21.15.60:0/299776215 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f1fa4005960 con 0x7f1fb412ece0 2023-08-24T10:09:57.487 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:57.487 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":308,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:09:57.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.487+0000 7f1fbb0ac640 1 -- 172.21.15.60:0/299776215 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1fa0037130 msgr2=0x7f1fa0039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:57.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.487+0000 7f1fbb0ac640 1 --2- 172.21.15.60:0/299776215 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1fa0037130 0x7f1fa0039640 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:57.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.487+0000 7f1fbb0ac640 1 -- 172.21.15.60:0/299776215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb412ece0 msgr2=0x7f1fb4129410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:57.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.487+0000 7f1fbb0ac640 1 --2- 172.21.15.60:0/299776215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb412ece0 0x7f1fb4129410 crc :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:57.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.487+0000 7f1fbb0ac640 1 -- 172.21.15.60:0/299776215 shutdown_connections 2023-08-24T10:09:57.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.487+0000 7f1fbb0ac640 1 --2- 172.21.15.60:0/299776215 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1fa0037130 0x7f1fa0039640 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:57.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.487+0000 7f1fbb0ac640 1 --2- 172.21.15.60:0/299776215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fb412ece0 0x7f1fb4129410 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:57.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.487+0000 7f1fbb0ac640 1 -- 172.21.15.60:0/299776215 >> 172.21.15.60:0/299776215 conn(0x7f1fb409e110 msgr2=0x7f1fb4097710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:57.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.487+0000 7f1fbb0ac640 1 -- 172.21.15.60:0/299776215 shutdown_connections 2023-08-24T10:09:57.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.487+0000 7f1fbb0ac640 1 -- 172.21.15.60:0/299776215 wait complete. 2023-08-24T10:09:57.499 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:09:57.500 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:09:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.607+0000 7f1279f9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:57.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.610+0000 7f1279f9e640 1 Processor -- start 2023-08-24T10:09:57.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.611+0000 7f1279f9e640 1 -- start start 2023-08-24T10:09:57.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.611+0000 7f1279f9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1274132b50 0x7f1274132f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:57.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.611+0000 7f1279f9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1274132610 con 0x7f127412f110 2023-08-24T10:09:57.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.611+0000 7f1279f9e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f127409c260 con 0x7f1274132b50 2023-08-24T10:09:57.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.611+0000 7f12737fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1274132b50 0x7f1274132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:57.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.611+0000 7f12737fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1274132b50 0x7f1274132f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54268/0 (socket says 172.21.15.60:54268) 2023-08-24T10:09:57.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.611+0000 7f1272ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f127412f110 0x7f127412f520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49804/0 (socket says 172.21.15.60:49804) 2023-08-24T10:09:57.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.611+0000 7f12737fe640 1 -- 172.21.15.60:0/1659282799 learned_addr learned my addr 172.21.15.60:0/1659282799 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:57.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.612+0000 7f12737fe640 1 -- 172.21.15.60:0/1659282799 >> v1:172.21.15.60:6789/0 conn(0x7f127412f110 legacy=0x7f127412f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:57.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.612+0000 7f12737fe640 1 -- 172.21.15.60:0/1659282799 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1274132840 con 0x7f1274132b50 2023-08-24T10:09:57.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.612+0000 7f12737fe640 1 --2- 172.21.15.60:0/1659282799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1274132b50 0x7f1274132f60 crc :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8e73fc30887eee server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:57.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.612+0000 7f12727fc640 1 -- 172.21.15.60:0/1659282799 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1264009dd0 con 0x7f1274132b50 2023-08-24T10:09:57.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.612+0000 7f12727fc640 1 -- 172.21.15.60:0/1659282799 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f126400e230 con 0x7f1274132b50 2023-08-24T10:09:57.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.612+0000 7f12727fc640 1 -- 172.21.15.60:0/1659282799 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f126400e4d0 con 0x7f1274132b50 2023-08-24T10:09:57.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.613+0000 7f1279f9e640 1 -- 172.21.15.60:0/1659282799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1274132b50 msgr2=0x7f1274132f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:57.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.613+0000 7f1279f9e640 1 --2- 172.21.15.60:0/1659282799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1274132b50 0x7f1274132f60 crc :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:57.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.613+0000 7f1279f9e640 1 -- 172.21.15.60:0/1659282799 shutdown_connections 2023-08-24T10:09:57.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.613+0000 7f1279f9e640 1 --2- 172.21.15.60:0/1659282799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1274132b50 0x7f1274132f60 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:57.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.613+0000 7f1279f9e640 1 -- 172.21.15.60:0/1659282799 >> 172.21.15.60:0/1659282799 conn(0x7f12740a26a0 msgr2=0x7f12740a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:57.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.613+0000 7f1279f9e640 1 -- 172.21.15.60:0/1659282799 shutdown_connections 2023-08-24T10:09:57.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.613+0000 7f1279f9e640 1 -- 172.21.15.60:0/1659282799 wait complete. 2023-08-24T10:09:57.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.613+0000 7f1279f9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:09:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.614+0000 7f1279f9e640 1 Processor -- start 2023-08-24T10:09:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.614+0000 7f1279f9e640 1 -- start start 2023-08-24T10:09:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.615+0000 7f1279f9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f127412f110 0x7f127409fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.615+0000 7f1279f9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127412cc80 con 0x7f127412f110 2023-08-24T10:09:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.615+0000 7f12737fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f127412f110 0x7f127409fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.615+0000 7f12737fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f127412f110 0x7f127409fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54284/0 (socket says 172.21.15.60:54284) 2023-08-24T10:09:57.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.615+0000 7f12737fe640 1 -- 172.21.15.60:0/1373128995 learned_addr learned my addr 172.21.15.60:0/1373128995 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:09:57.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.615+0000 7f12737fe640 1 -- 172.21.15.60:0/1373128995 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127409f400 con 0x7f127412f110 2023-08-24T10:09:57.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.615+0000 7f12737fe640 1 --2- 172.21.15.60:0/1373128995 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f127412f110 0x7f127409fcd0 crc :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:57.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.615+0000 7f125bfff640 1 -- 172.21.15.60:0/1373128995 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1264014410 con 0x7f127412f110 2023-08-24T10:09:57.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.615+0000 7f125bfff640 1 -- 172.21.15.60:0/1373128995 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1264014570 con 0x7f127412f110 2023-08-24T10:09:57.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.615+0000 7f1279f9e640 1 -- 172.21.15.60:0/1373128995 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12740a18e0 con 0x7f127412f110 2023-08-24T10:09:57.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.615+0000 7f125bfff640 1 -- 172.21.15.60:0/1373128995 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1264014810 con 0x7f127412f110 2023-08-24T10:09:57.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.615+0000 7f1279f9e640 1 -- 172.21.15.60:0/1373128995 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f127406a1d0 con 0x7f127412f110 2023-08-24T10:09:57.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.616+0000 7f125bfff640 1 -- 172.21.15.60:0/1373128995 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f12640149c0 con 0x7f127412f110 2023-08-24T10:09:57.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.616+0000 7f1279f9e640 1 -- 172.21.15.60:0/1373128995 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f127411c2b0 con 0x7f127412f110 2023-08-24T10:09:57.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.616+0000 7f125bfff640 1 --2- 172.21.15.60:0/1373128995 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1260037130 0x7f1260039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:09:57.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.617+0000 7f125bfff640 1 -- 172.21.15.60:0/1373128995 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f1264019040 con 0x7f127412f110 2023-08-24T10:09:57.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.617+0000 7f1272ffd640 1 --2- 172.21.15.60:0/1373128995 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1260037130 0x7f1260039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:09:57.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.617+0000 7f1272ffd640 1 --2- 172.21.15.60:0/1373128995 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1260037130 0x7f1260039640 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:09:57.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.621+0000 7f125bfff640 1 -- 172.21.15.60:0/1373128995 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f126400f8f0 con 0x7f127412f110 2023-08-24T10:09:57.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.793+0000 7f1279f9e640 1 -- 172.21.15.60:0/1373128995 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f12741334a0 con 0x7f1260037130 2023-08-24T10:09:57.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.794+0000 7f125bfff640 1 -- 172.21.15.60:0/1373128995 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f12680070b0 con 0x7f1260037130 2023-08-24T10:09:57.797 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:09:57.797 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:09:57.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.796+0000 7f1279f9e640 1 -- 172.21.15.60:0/1373128995 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1260037130 msgr2=0x7f1260039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:57.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.796+0000 7f1279f9e640 1 --2- 172.21.15.60:0/1373128995 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1260037130 0x7f1260039640 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:57.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.796+0000 7f1279f9e640 1 -- 172.21.15.60:0/1373128995 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f127412f110 msgr2=0x7f127409fcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:09:57.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.796+0000 7f1279f9e640 1 --2- 172.21.15.60:0/1373128995 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f127412f110 0x7f127409fcd0 crc :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:57.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.796+0000 7f1279f9e640 1 -- 172.21.15.60:0/1373128995 shutdown_connections 2023-08-24T10:09:57.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.796+0000 7f1279f9e640 1 --2- 172.21.15.60:0/1373128995 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f1260037130 0x7f1260039640 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:57.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.796+0000 7f1279f9e640 1 --2- 172.21.15.60:0/1373128995 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f127412f110 0x7f127409fcd0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:09:57.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.796+0000 7f1279f9e640 1 -- 172.21.15.60:0/1373128995 >> 172.21.15.60:0/1373128995 conn(0x7f12740a26a0 msgr2=0x7f12741ba920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:09:57.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.796+0000 7f1279f9e640 1 -- 172.21.15.60:0/1373128995 shutdown_connections 2023-08-24T10:09:57.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:09:57.796+0000 7f1279f9e640 1 -- 172.21.15.60:0/1373128995 wait complete. 2023-08-24T10:09:57.808 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2023-08-24T10:09:57.552690+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:09:58.871 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~291s 2023-08-24T10:09:58.871 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~291s 2023-08-24T10:09:58.871 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~291s 2023-08-24T10:09:58.872 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~291s 2023-08-24T10:10:00.810 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:10:00.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.930+0000 7f2383aac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:10:00.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.934+0000 7f2383aac640 1 Processor -- start 2023-08-24T10:10:00.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.934+0000 7f2383aac640 1 -- start start 2023-08-24T10:10:00.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.935+0000 7f2383aac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f237c132630 0x7f237c134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:00.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.935+0000 7f2383aac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237c06e320 con 0x7f237c096860 2023-08-24T10:10:00.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.935+0000 7f2383aac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237c06db70 con 0x7f237c132630 2023-08-24T10:10:00.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.935+0000 7f2381047640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f237c132630 0x7f237c134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:00.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.935+0000 7f2381047640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f237c132630 0x7f237c134a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54290/0 (socket says 172.21.15.60:54290) 2023-08-24T10:10:00.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.935+0000 7f2381047640 1 -- 172.21.15.60:0/383416362 learned_addr learned my addr 172.21.15.60:0/383416362 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:10:00.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.936+0000 7f2381047640 1 -- 172.21.15.60:0/383416362 >> v1:172.21.15.60:6789/0 conn(0x7f237c096860 legacy=0x7f237c096c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:00.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.936+0000 7f2381047640 1 -- 172.21.15.60:0/383416362 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237c06cef0 con 0x7f237c132630 2023-08-24T10:10:00.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.936+0000 7f2381047640 1 --2- 172.21.15.60:0/383416362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f237c132630 0x7f237c134a50 crc :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac6beaa31072990 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:00.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.936+0000 7f2380846640 1 -- 172.21.15.60:0/383416362 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f236c009dd0 con 0x7f237c132630 2023-08-24T10:10:00.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.936+0000 7f2380846640 1 -- 172.21.15.60:0/383416362 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f236c00e230 con 0x7f237c132630 2023-08-24T10:10:00.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.936+0000 7f2380846640 1 -- 172.21.15.60:0/383416362 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f236c00e4d0 con 0x7f237c132630 2023-08-24T10:10:00.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.936+0000 7f2383aac640 1 -- 172.21.15.60:0/383416362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f237c132630 msgr2=0x7f237c134a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:00.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.936+0000 7f2383aac640 1 --2- 172.21.15.60:0/383416362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f237c132630 0x7f237c134a50 crc :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:00.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.937+0000 7f2383aac640 1 -- 172.21.15.60:0/383416362 shutdown_connections 2023-08-24T10:10:00.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.937+0000 7f2383aac640 1 --2- 172.21.15.60:0/383416362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f237c132630 0x7f237c134a50 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:00.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.937+0000 7f2383aac640 1 -- 172.21.15.60:0/383416362 >> 172.21.15.60:0/383416362 conn(0x7f237c09e110 msgr2=0x7f237c09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:10:00.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.937+0000 7f2383aac640 1 -- 172.21.15.60:0/383416362 shutdown_connections 2023-08-24T10:10:00.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.937+0000 7f2383aac640 1 -- 172.21.15.60:0/383416362 wait complete. 2023-08-24T10:10:00.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.937+0000 7f2383aac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:10:00.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.938+0000 7f2383aac640 1 Processor -- start 2023-08-24T10:10:00.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.938+0000 7f2383aac640 1 -- start start 2023-08-24T10:10:00.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.939+0000 7f2383aac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f237c096860 0x7f237c127b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:00.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.939+0000 7f2383aac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c129530 con 0x7f237c096860 2023-08-24T10:10:00.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.939+0000 7f2381848640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f237c096860 0x7f237c127b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:00.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.939+0000 7f2381848640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f237c096860 0x7f237c127b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54292/0 (socket says 172.21.15.60:54292) 2023-08-24T10:10:00.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.939+0000 7f2381848640 1 -- 172.21.15.60:0/2647088955 learned_addr learned my addr 172.21.15.60:0/2647088955 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:10:00.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.939+0000 7f2381848640 1 -- 172.21.15.60:0/2647088955 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237c1290c0 con 0x7f237c096860 2023-08-24T10:10:00.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.939+0000 7f2381848640 1 --2- 172.21.15.60:0/2647088955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f237c096860 0x7f237c127b60 crc :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:00.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.939+0000 7f23727fc640 1 -- 172.21.15.60:0/2647088955 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f237800cb60 con 0x7f237c096860 2023-08-24T10:10:00.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.940+0000 7f23727fc640 1 -- 172.21.15.60:0/2647088955 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f237800ccc0 con 0x7f237c096860 2023-08-24T10:10:00.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.940+0000 7f2383aac640 1 -- 172.21.15.60:0/2647088955 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237c12efa0 con 0x7f237c096860 2023-08-24T10:10:00.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.940+0000 7f23727fc640 1 -- 172.21.15.60:0/2647088955 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f237800f230 con 0x7f237c096860 2023-08-24T10:10:00.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.940+0000 7f2383aac640 1 -- 172.21.15.60:0/2647088955 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237c097fa0 con 0x7f237c096860 2023-08-24T10:10:00.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.940+0000 7f23727fc640 1 -- 172.21.15.60:0/2647088955 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f237800f3e0 con 0x7f237c096860 2023-08-24T10:10:00.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.940+0000 7f23727fc640 1 --2- 172.21.15.60:0/2647088955 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2360037130 0x7f2360039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:00.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.941+0000 7f23727fc640 1 -- 172.21.15.60:0/2647088955 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f2378046d50 con 0x7f237c096860 2023-08-24T10:10:00.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.941+0000 7f2383aac640 1 -- 172.21.15.60:0/2647088955 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f237c061800 con 0x7f237c096860 2023-08-24T10:10:00.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.941+0000 7f2381047640 1 --2- 172.21.15.60:0/2647088955 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2360037130 0x7f2360039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:00.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.941+0000 7f2381047640 1 --2- 172.21.15.60:0/2647088955 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2360037130 0x7f2360039640 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:00.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:00.946+0000 7f23727fc640 1 -- 172.21.15.60:0/2647088955 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f237800ace0 con 0x7f237c096860 2023-08-24T10:10:01.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.119+0000 7f2383aac640 1 -- 172.21.15.60:0/2647088955 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f237c06d190 con 0x7f2360037130 2023-08-24T10:10:01.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.120+0000 7f23727fc640 1 -- 172.21.15.60:0/2647088955 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f236c009df0 con 0x7f2360037130 2023-08-24T10:10:01.122 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:10:01.123 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:10:01.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.122+0000 7f2383aac640 1 -- 172.21.15.60:0/2647088955 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2360037130 msgr2=0x7f2360039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:01.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.122+0000 7f2383aac640 1 --2- 172.21.15.60:0/2647088955 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2360037130 0x7f2360039640 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:01.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.122+0000 7f2383aac640 1 -- 172.21.15.60:0/2647088955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f237c096860 msgr2=0x7f237c127b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:01.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.122+0000 7f2383aac640 1 --2- 172.21.15.60:0/2647088955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f237c096860 0x7f237c127b60 crc :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:01.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.122+0000 7f2383aac640 1 -- 172.21.15.60:0/2647088955 shutdown_connections 2023-08-24T10:10:01.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.122+0000 7f2383aac640 1 --2- 172.21.15.60:0/2647088955 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2360037130 0x7f2360039640 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:01.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.122+0000 7f2383aac640 1 --2- 172.21.15.60:0/2647088955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f237c096860 0x7f237c127b60 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:01.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.122+0000 7f2383aac640 1 -- 172.21.15.60:0/2647088955 >> 172.21.15.60:0/2647088955 conn(0x7f237c09e110 msgr2=0x7f237c0950c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:10:01.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.122+0000 7f2383aac640 1 -- 172.21.15.60:0/2647088955 shutdown_connections 2023-08-24T10:10:01.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.122+0000 7f2383aac640 1 -- 172.21.15.60:0/2647088955 wait complete. 2023-08-24T10:10:01.135 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2023-08-24T10:09:59.552964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:10:01.136 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:10:01.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.242+0000 7fc7bcc52640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:10:01.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.246+0000 7fc7bcc52640 1 Processor -- start 2023-08-24T10:10:01.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.246+0000 7fc7bcc52640 1 -- start start 2023-08-24T10:10:01.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.247+0000 7fc7bcc52640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7b8070110 0x7fc7b81356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:01.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.247+0000 7fc7bcc52640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b8096f10 con 0x7fc7b8071e40 2023-08-24T10:10:01.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.247+0000 7fc7bcc52640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7b80977f0 con 0x7fc7b8070110 2023-08-24T10:10:01.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.247+0000 7fc7affff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7b8070110 0x7fc7b81356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:01.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.247+0000 7fc7affff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7b8070110 0x7fc7b81356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54308/0 (socket says 172.21.15.60:54308) 2023-08-24T10:10:01.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.247+0000 7fc7affff640 1 -- 172.21.15.60:0/2065574430 learned_addr learned my addr 172.21.15.60:0/2065574430 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:10:01.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.247+0000 7fc7b659c640 1 --1- 172.21.15.60:0/2065574430 >> v1:172.21.15.60:6789/0 conn(0x7fc7b8071e40 0x7fc7b8096800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49832/0 (socket says 172.21.15.60:49832) 2023-08-24T10:10:01.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.247+0000 7fc7affff640 1 -- 172.21.15.60:0/2065574430 >> v1:172.21.15.60:6789/0 conn(0x7fc7b8071e40 legacy=0x7fc7b8096800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:01.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.247+0000 7fc7affff640 1 -- 172.21.15.60:0/2065574430 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b809a280 con 0x7fc7b8070110 2023-08-24T10:10:01.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.247+0000 7fc7affff640 1 --2- 172.21.15.60:0/2065574430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7b8070110 0x7fc7b81356a0 crc :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24f51c221c7b2d server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:01.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.247+0000 7fc7b5d9b640 1 -- 172.21.15.60:0/2065574430 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc7a0009dd0 con 0x7fc7b8070110 2023-08-24T10:10:01.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.248+0000 7fc7b5d9b640 1 -- 172.21.15.60:0/2065574430 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc7a000e230 con 0x7fc7b8070110 2023-08-24T10:10:01.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.248+0000 7fc7b5d9b640 1 -- 172.21.15.60:0/2065574430 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc7a000e4d0 con 0x7fc7b8070110 2023-08-24T10:10:01.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.248+0000 7fc7bcc52640 1 -- 172.21.15.60:0/2065574430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7b8070110 msgr2=0x7fc7b81356a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:01.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.248+0000 7fc7bcc52640 1 --2- 172.21.15.60:0/2065574430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7b8070110 0x7fc7b81356a0 crc :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:01.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.248+0000 7fc7bcc52640 1 -- 172.21.15.60:0/2065574430 shutdown_connections 2023-08-24T10:10:01.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.248+0000 7fc7bcc52640 1 --2- 172.21.15.60:0/2065574430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7b8070110 0x7fc7b81356a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:01.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.248+0000 7fc7bcc52640 1 -- 172.21.15.60:0/2065574430 >> 172.21.15.60:0/2065574430 conn(0x7fc7b809e110 msgr2=0x7fc7b809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:10:01.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.248+0000 7fc7bcc52640 1 -- 172.21.15.60:0/2065574430 shutdown_connections 2023-08-24T10:10:01.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.248+0000 7fc7bcc52640 1 -- 172.21.15.60:0/2065574430 wait complete. 2023-08-24T10:10:01.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.249+0000 7fc7bcc52640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:10:01.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.250+0000 7fc7bcc52640 1 Processor -- start 2023-08-24T10:10:01.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.250+0000 7fc7bcc52640 1 -- start start 2023-08-24T10:10:01.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.250+0000 7fc7bcc52640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7b8070110 0x7fc7b8093460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:01.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.250+0000 7fc7bcc52640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b808ff10 con 0x7fc7b8070110 2023-08-24T10:10:01.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.250+0000 7fc7b659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7b8070110 0x7fc7b8093460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:01.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.250+0000 7fc7b659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7b8070110 0x7fc7b8093460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54320/0 (socket says 172.21.15.60:54320) 2023-08-24T10:10:01.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.250+0000 7fc7b659c640 1 -- 172.21.15.60:0/781452719 learned_addr learned my addr 172.21.15.60:0/781452719 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:10:01.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.251+0000 7fc7b659c640 1 -- 172.21.15.60:0/781452719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b8090680 con 0x7fc7b8070110 2023-08-24T10:10:01.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.251+0000 7fc7b659c640 1 --2- 172.21.15.60:0/781452719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7b8070110 0x7fc7b8093460 crc :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:01.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.251+0000 7fc7aeffd640 1 -- 172.21.15.60:0/781452719 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc7a80071e0 con 0x7fc7b8070110 2023-08-24T10:10:01.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.251+0000 7fc7aeffd640 1 -- 172.21.15.60:0/781452719 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc7a8007340 con 0x7fc7b8070110 2023-08-24T10:10:01.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.251+0000 7fc7bcc52640 1 -- 172.21.15.60:0/781452719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b8092e80 con 0x7fc7b8070110 2023-08-24T10:10:01.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.251+0000 7fc7aeffd640 1 -- 172.21.15.60:0/781452719 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc7a80075e0 con 0x7fc7b8070110 2023-08-24T10:10:01.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.251+0000 7fc7bcc52640 1 -- 172.21.15.60:0/781452719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b8092b70 con 0x7fc7b8070110 2023-08-24T10:10:01.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.252+0000 7fc7aeffd640 1 -- 172.21.15.60:0/781452719 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7fc7a80057e0 con 0x7fc7b8070110 2023-08-24T10:10:01.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.252+0000 7fc7bcc52640 1 -- 172.21.15.60:0/781452719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b8061800 con 0x7fc7b8070110 2023-08-24T10:10:01.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.252+0000 7fc7aeffd640 1 --2- 172.21.15.60:0/781452719 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc798037130 0x7fc798039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:01.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.252+0000 7fc7aeffd640 1 -- 172.21.15.60:0/781452719 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fc7a80463b0 con 0x7fc7b8070110 2023-08-24T10:10:01.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.252+0000 7fc7affff640 1 --2- 172.21.15.60:0/781452719 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc798037130 0x7fc798039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:01.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.253+0000 7fc7affff640 1 --2- 172.21.15.60:0/781452719 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc798037130 0x7fc798039640 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:01.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.257+0000 7fc7aeffd640 1 -- 172.21.15.60:0/781452719 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7fc7a800d330 con 0x7fc7b8070110 2023-08-24T10:10:01.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.430+0000 7fc7bcc52640 1 -- 172.21.15.60:0/781452719 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7b80932f0 con 0x7fc798037130 2023-08-24T10:10:01.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.432+0000 7fc7aeffd640 1 -- 172.21.15.60:0/781452719 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7fc7b80932f0 con 0x7fc798037130 2023-08-24T10:10:01.434 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:10:01.434 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:10:01.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.434+0000 7fc7bcc52640 1 -- 172.21.15.60:0/781452719 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc798037130 msgr2=0x7fc798039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:01.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.434+0000 7fc7bcc52640 1 --2- 172.21.15.60:0/781452719 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc798037130 0x7fc798039640 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:01.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.434+0000 7fc7bcc52640 1 -- 172.21.15.60:0/781452719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7b8070110 msgr2=0x7fc7b8093460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:01.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.434+0000 7fc7bcc52640 1 --2- 172.21.15.60:0/781452719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7b8070110 0x7fc7b8093460 crc :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:01.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.434+0000 7fc7bcc52640 1 -- 172.21.15.60:0/781452719 shutdown_connections 2023-08-24T10:10:01.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.434+0000 7fc7bcc52640 1 --2- 172.21.15.60:0/781452719 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7fc798037130 0x7fc798039640 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:01.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.434+0000 7fc7bcc52640 1 --2- 172.21.15.60:0/781452719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7b8070110 0x7fc7b8093460 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:01.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.434+0000 7fc7bcc52640 1 -- 172.21.15.60:0/781452719 >> 172.21.15.60:0/781452719 conn(0x7fc7b809e110 msgr2=0x7fc7b8098490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:10:01.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.434+0000 7fc7bcc52640 1 -- 172.21.15.60:0/781452719 shutdown_connections 2023-08-24T10:10:01.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.434+0000 7fc7bcc52640 1 -- 172.21.15.60:0/781452719 wait complete. 2023-08-24T10:10:01.447 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2023-08-24T10:09:59.552964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:10:01.448 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:10:01.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:10:01.449 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:10:01.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:10:01.450 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:10:01.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:10:01.450 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:10:01.451 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:10:01.451 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:10:01.451 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:10:01.451 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:10:01.451 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:10:01.452 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:10:01.452 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:10:01.452 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:10:01.452 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:10:01.453 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:10:01.453 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:10:01.453 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:10:01.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.560+0000 7f5377b9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:10:01.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.563+0000 7f5377b9e640 1 Processor -- start 2023-08-24T10:10:01.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.563+0000 7f5377b9e640 1 -- start start 2023-08-24T10:10:01.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.564+0000 7f5377b9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5370070c30 0x7f5370071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:01.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.564+0000 7f5377b9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537006dfc0 con 0x7f53700990b0 2023-08-24T10:10:01.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.564+0000 7f5377b9e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f537006d9b0 con 0x7f5370070c30 2023-08-24T10:10:01.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.564+0000 7f537593a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5370070c30 0x7f5370071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:01.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.564+0000 7f537593a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5370070c30 0x7f5370071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54336/0 (socket says 172.21.15.60:54336) 2023-08-24T10:10:01.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.564+0000 7f537593a640 1 -- 172.21.15.60:0/2040651764 learned_addr learned my addr 172.21.15.60:0/2040651764 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:10:01.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.564+0000 7f537593a640 1 -- 172.21.15.60:0/2040651764 >> v1:172.21.15.60:6789/0 conn(0x7f53700990b0 legacy=0x7f53700994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:01.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.564+0000 7f537593a640 1 -- 172.21.15.60:0/2040651764 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537006e6e0 con 0x7f5370070c30 2023-08-24T10:10:01.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.564+0000 7f537593a640 1 --2- 172.21.15.60:0/2040651764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370070c30 0x7f5370071040 crc :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b229612eace39c server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:01.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.564+0000 7f5374938640 1 -- 172.21.15.60:0/2040651764 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5364009dd0 con 0x7f5370070c30 2023-08-24T10:10:01.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.564+0000 7f5374938640 1 -- 172.21.15.60:0/2040651764 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f536400e230 con 0x7f5370070c30 2023-08-24T10:10:01.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.564+0000 7f5374938640 1 -- 172.21.15.60:0/2040651764 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f536400e4d0 con 0x7f5370070c30 2023-08-24T10:10:01.569 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.565+0000 7f5377b9e640 1 -- 172.21.15.60:0/2040651764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370070c30 msgr2=0x7f5370071040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:01.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.565+0000 7f5377b9e640 1 --2- 172.21.15.60:0/2040651764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370070c30 0x7f5370071040 crc :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:01.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.565+0000 7f5377b9e640 1 -- 172.21.15.60:0/2040651764 shutdown_connections 2023-08-24T10:10:01.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.565+0000 7f5377b9e640 1 --2- 172.21.15.60:0/2040651764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370070c30 0x7f5370071040 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:01.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.565+0000 7f5377b9e640 1 -- 172.21.15.60:0/2040651764 >> 172.21.15.60:0/2040651764 conn(0x7f537009e110 msgr2=0x7f537009e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:10:01.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.565+0000 7f5377b9e640 1 -- 172.21.15.60:0/2040651764 shutdown_connections 2023-08-24T10:10:01.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.565+0000 7f5377b9e640 1 -- 172.21.15.60:0/2040651764 wait complete. 2023-08-24T10:10:01.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.565+0000 7f5377b9e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:10:01.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.566+0000 7f5377b9e640 1 Processor -- start 2023-08-24T10:10:01.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.566+0000 7f5377b9e640 1 -- start start 2023-08-24T10:10:01.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.567+0000 7f5377b9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370070c30 0x7f53701c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:01.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.567+0000 7f5377b9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537012b3e0 con 0x7f5370070c30 2023-08-24T10:10:01.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.567+0000 7f537593a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370070c30 0x7f53701c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:01.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.567+0000 7f537593a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370070c30 0x7f53701c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54340/0 (socket says 172.21.15.60:54340) 2023-08-24T10:10:01.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.567+0000 7f537593a640 1 -- 172.21.15.60:0/148240126 learned_addr learned my addr 172.21.15.60:0/148240126 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:10:01.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.567+0000 7f537593a640 1 -- 172.21.15.60:0/148240126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53701c0930 con 0x7f5370070c30 2023-08-24T10:10:01.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.567+0000 7f537593a640 1 --2- 172.21.15.60:0/148240126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370070c30 0x7f53701c0fa0 crc :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:01.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.568+0000 7f53627fc640 1 -- 172.21.15.60:0/148240126 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5364014540 con 0x7f5370070c30 2023-08-24T10:10:01.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.568+0000 7f53627fc640 1 -- 172.21.15.60:0/148240126 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f53640146a0 con 0x7f5370070c30 2023-08-24T10:10:01.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.568+0000 7f5377b9e640 1 -- 172.21.15.60:0/148240126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53701c0c00 con 0x7f5370070c30 2023-08-24T10:10:01.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.568+0000 7f53627fc640 1 -- 172.21.15.60:0/148240126 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5364014940 con 0x7f5370070c30 2023-08-24T10:10:01.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.568+0000 7f5377b9e640 1 -- 172.21.15.60:0/148240126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f537012c980 con 0x7f5370070c30 2023-08-24T10:10:01.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.568+0000 7f5377b9e640 1 -- 172.21.15.60:0/148240126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f537012dd10 con 0x7f5370070c30 2023-08-24T10:10:01.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.569+0000 7f53627fc640 1 -- 172.21.15.60:0/148240126 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f5364014af0 con 0x7f5370070c30 2023-08-24T10:10:01.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.569+0000 7f53627fc640 1 --2- 172.21.15.60:0/148240126 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5354037130 0x7f5354039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:01.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.569+0000 7f53627fc640 1 -- 172.21.15.60:0/148240126 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f5364019040 con 0x7f5370070c30 2023-08-24T10:10:01.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.569+0000 7f5375139640 1 --2- 172.21.15.60:0/148240126 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5354037130 0x7f5354039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:01.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.569+0000 7f5375139640 1 --2- 172.21.15.60:0/148240126 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5354037130 0x7f5354039640 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:01.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.573+0000 7f53627fc640 1 -- 172.21.15.60:0/148240126 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f5364018190 con 0x7f5370070c30 2023-08-24T10:10:01.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.816+0000 7f5377b9e640 1 -- 172.21.15.60:0/148240126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f537012c1b0 con 0x7f5370070c30 2023-08-24T10:10:01.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.816+0000 7f53627fc640 1 -- 172.21.15.60:0/148240126 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f536400fa90 con 0x7f5370070c30 2023-08-24T10:10:01.818 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:10:01.819 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":312,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:10:01.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.818+0000 7f5377b9e640 1 -- 172.21.15.60:0/148240126 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5354037130 msgr2=0x7f5354039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:01.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.818+0000 7f5377b9e640 1 --2- 172.21.15.60:0/148240126 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5354037130 0x7f5354039640 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:01.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.818+0000 7f5377b9e640 1 -- 172.21.15.60:0/148240126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370070c30 msgr2=0x7f53701c0fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:01.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.818+0000 7f5377b9e640 1 --2- 172.21.15.60:0/148240126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370070c30 0x7f53701c0fa0 crc :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:01.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.818+0000 7f5377b9e640 1 -- 172.21.15.60:0/148240126 shutdown_connections 2023-08-24T10:10:01.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.818+0000 7f5377b9e640 1 --2- 172.21.15.60:0/148240126 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5354037130 0x7f5354039640 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:01.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.818+0000 7f5377b9e640 1 --2- 172.21.15.60:0/148240126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370070c30 0x7f53701c0fa0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:01.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.818+0000 7f5377b9e640 1 -- 172.21.15.60:0/148240126 >> 172.21.15.60:0/148240126 conn(0x7f537009e110 msgr2=0x7f53701b75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:10:01.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.818+0000 7f5377b9e640 1 -- 172.21.15.60:0/148240126 shutdown_connections 2023-08-24T10:10:01.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.818+0000 7f5377b9e640 1 -- 172.21.15.60:0/148240126 wait complete. 2023-08-24T10:10:01.831 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:10:01.831 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:10:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.938+0000 7f534971e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:10:01.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.942+0000 7f534971e640 1 Processor -- start 2023-08-24T10:10:01.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.942+0000 7f534971e640 1 -- start start 2023-08-24T10:10:01.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.943+0000 7f534971e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5344132630 0x7f5344134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:01.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.943+0000 7f534971e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f534406e320 con 0x7f5344096860 2023-08-24T10:10:01.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.943+0000 7f534971e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f534406db70 con 0x7f5344132630 2023-08-24T10:10:01.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.943+0000 7f5342ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5344096860 0x7f5344096c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49846/0 (socket says 172.21.15.60:49846) 2023-08-24T10:10:01.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.943+0000 7f5342ffd640 1 -- 172.21.15.60:0/1203151364 learned_addr learned my addr 172.21.15.60:0/1203151364 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:10:01.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.943+0000 7f533bfff640 1 --2- 172.21.15.60:0/1203151364 >> v2:172.21.15.60:3300/0 conn(0x7f5344132630 0x7f5344134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:01.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.944+0000 7f533bfff640 1 -- 172.21.15.60:0/1203151364 >> v1:172.21.15.60:6789/0 conn(0x7f5344096860 legacy=0x7f5344096c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:01.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.944+0000 7f533bfff640 1 -- 172.21.15.60:0/1203151364 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534406cef0 con 0x7f5344132630 2023-08-24T10:10:01.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.944+0000 7f533bfff640 1 --2- 172.21.15.60:0/1203151364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5344132630 0x7f5344134a50 crc :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f60f2280df1cf95 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:01.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.944+0000 7f53427fc640 1 -- 172.21.15.60:0/1203151364 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f532c009ce0 con 0x7f5344132630 2023-08-24T10:10:01.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.944+0000 7f53427fc640 1 -- 172.21.15.60:0/1203151364 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f532c009e40 con 0x7f5344132630 2023-08-24T10:10:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.944+0000 7f53427fc640 1 -- 172.21.15.60:0/1203151364 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f532c00e360 con 0x7f5344132630 2023-08-24T10:10:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.944+0000 7f534971e640 1 -- 172.21.15.60:0/1203151364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5344132630 msgr2=0x7f5344134a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.944+0000 7f534971e640 1 --2- 172.21.15.60:0/1203151364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5344132630 0x7f5344134a50 crc :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.945+0000 7f534971e640 1 -- 172.21.15.60:0/1203151364 shutdown_connections 2023-08-24T10:10:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.945+0000 7f534971e640 1 --2- 172.21.15.60:0/1203151364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5344132630 0x7f5344134a50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.945+0000 7f534971e640 1 -- 172.21.15.60:0/1203151364 >> 172.21.15.60:0/1203151364 conn(0x7f534409e110 msgr2=0x7f534409e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:10:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.945+0000 7f534971e640 1 -- 172.21.15.60:0/1203151364 shutdown_connections 2023-08-24T10:10:01.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.945+0000 7f534971e640 1 -- 172.21.15.60:0/1203151364 wait complete. 2023-08-24T10:10:01.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.945+0000 7f534971e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:10:01.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.946+0000 7f534971e640 1 Processor -- start 2023-08-24T10:10:01.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.946+0000 7f534971e640 1 -- start start 2023-08-24T10:10:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.946+0000 7f534971e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5344096860 0x7f53441c1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.946+0000 7f534971e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534408e680 con 0x7f5344096860 2023-08-24T10:10:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.947+0000 7f5342ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5344096860 0x7f53441c1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.947+0000 7f5342ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5344096860 0x7f53441c1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54368/0 (socket says 172.21.15.60:54368) 2023-08-24T10:10:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.947+0000 7f5342ffd640 1 -- 172.21.15.60:0/2623676373 learned_addr learned my addr 172.21.15.60:0/2623676373 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:10:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.947+0000 7f5342ffd640 1 -- 172.21.15.60:0/2623676373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5344090c20 con 0x7f5344096860 2023-08-24T10:10:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.947+0000 7f5342ffd640 1 --2- 172.21.15.60:0/2623676373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5344096860 0x7f53441c1d80 crc :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.947+0000 7f533b7fe640 1 -- 172.21.15.60:0/2623676373 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f53340059d0 con 0x7f5344096860 2023-08-24T10:10:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.947+0000 7f533b7fe640 1 -- 172.21.15.60:0/2623676373 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5334005b30 con 0x7f5344096860 2023-08-24T10:10:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.947+0000 7f534971e640 1 -- 172.21.15.60:0/2623676373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53441c19e0 con 0x7f5344096860 2023-08-24T10:10:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.947+0000 7f533b7fe640 1 -- 172.21.15.60:0/2623676373 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5334005dd0 con 0x7f5344096860 2023-08-24T10:10:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.947+0000 7f534971e640 1 -- 172.21.15.60:0/2623676373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534408f720 con 0x7f5344096860 2023-08-24T10:10:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.948+0000 7f533b7fe640 1 -- 172.21.15.60:0/2623676373 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f53340070b0 con 0x7f5344096860 2023-08-24T10:10:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.948+0000 7f533b7fe640 1 --2- 172.21.15.60:0/2623676373 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5324037130 0x7f5324039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:01.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.948+0000 7f533b7fe640 1 -- 172.21.15.60:0/2623676373 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f5334046470 con 0x7f5344096860 2023-08-24T10:10:01.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.948+0000 7f534971e640 1 -- 172.21.15.60:0/2623676373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53441c1bd0 con 0x7f5344096860 2023-08-24T10:10:01.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.948+0000 7f533bfff640 1 --2- 172.21.15.60:0/2623676373 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5324037130 0x7f5324039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:01.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.949+0000 7f533bfff640 1 --2- 172.21.15.60:0/2623676373 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5324037130 0x7f5324039640 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:01.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:01.953+0000 7f533b7fe640 1 -- 172.21.15.60:0/2623676373 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f533400d330 con 0x7f5344096860 2023-08-24T10:10:02.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:02.125+0000 7f534971e640 1 -- 172.21.15.60:0/2623676373 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f534406d190 con 0x7f5324037130 2023-08-24T10:10:02.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:02.126+0000 7f533b7fe640 1 -- 172.21.15.60:0/2623676373 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f532c0124e0 con 0x7f5324037130 2023-08-24T10:10:02.128 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:10:02.129 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:10:02.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:02.128+0000 7f534971e640 1 -- 172.21.15.60:0/2623676373 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5324037130 msgr2=0x7f5324039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:02.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:02.128+0000 7f534971e640 1 --2- 172.21.15.60:0/2623676373 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5324037130 0x7f5324039640 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:02.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:02.128+0000 7f534971e640 1 -- 172.21.15.60:0/2623676373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5344096860 msgr2=0x7f53441c1d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:02.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:02.128+0000 7f534971e640 1 --2- 172.21.15.60:0/2623676373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5344096860 0x7f53441c1d80 crc :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:02.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:02.128+0000 7f534971e640 1 -- 172.21.15.60:0/2623676373 shutdown_connections 2023-08-24T10:10:02.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:02.128+0000 7f534971e640 1 --2- 172.21.15.60:0/2623676373 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f5324037130 0x7f5324039640 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:02.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:02.128+0000 7f534971e640 1 --2- 172.21.15.60:0/2623676373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5344096860 0x7f53441c1d80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:02.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:02.128+0000 7f534971e640 1 -- 172.21.15.60:0/2623676373 >> 172.21.15.60:0/2623676373 conn(0x7f534409e110 msgr2=0x7f5344131180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:10:02.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:02.128+0000 7f534971e640 1 -- 172.21.15.60:0/2623676373 shutdown_connections 2023-08-24T10:10:02.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:02.128+0000 7f534971e640 1 -- 172.21.15.60:0/2623676373 wait complete. 2023-08-24T10:10:02.140 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2023-08-24T10:10:01.553299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:10:04.074 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~297s 2023-08-24T10:10:04.075 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~297s 2023-08-24T10:10:04.075 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~297s 2023-08-24T10:10:04.075 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~297s 2023-08-24T10:10:05.141 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:10:05.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.261+0000 7f3fce59e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:10:05.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.265+0000 7f3fce59e640 1 Processor -- start 2023-08-24T10:10:05.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.265+0000 7f3fce59e640 1 -- start start 2023-08-24T10:10:05.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.266+0000 7f3fce59e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3fc8131d40 0x7f3fc8134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:05.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.266+0000 7f3fce59e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc8099440 con 0x7f3fc806d300 2023-08-24T10:10:05.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.266+0000 7f3fce59e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc809a050 con 0x7f3fc8131d40 2023-08-24T10:10:05.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.266+0000 7f3fc7fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3fc806d300 0x7f3fc812bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49862/0 (socket says 172.21.15.60:49862) 2023-08-24T10:10:05.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.266+0000 7f3fc7fff640 1 -- 172.21.15.60:0/538641900 learned_addr learned my addr 172.21.15.60:0/538641900 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:10:05.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.266+0000 7f3fc77fe640 1 --2- 172.21.15.60:0/538641900 >> v2:172.21.15.60:3300/0 conn(0x7f3fc8131d40 0x7f3fc8134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:05.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.266+0000 7f3fc77fe640 1 -- 172.21.15.60:0/538641900 >> v1:172.21.15.60:6789/0 conn(0x7f3fc806d300 legacy=0x7f3fc812bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:05.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.266+0000 7f3fc77fe640 1 -- 172.21.15.60:0/538641900 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc812b9d0 con 0x7f3fc8131d40 2023-08-24T10:10:05.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.267+0000 7f3fc77fe640 1 --2- 172.21.15.60:0/538641900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc8131d40 0x7f3fc8134160 crc :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cfd207d2909a8a server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:05.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.267+0000 7f3fc6ffd640 1 -- 172.21.15.60:0/538641900 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3fb8009cd0 con 0x7f3fc8131d40 2023-08-24T10:10:05.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.267+0000 7f3fc6ffd640 1 -- 172.21.15.60:0/538641900 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3fb8009e30 con 0x7f3fc8131d40 2023-08-24T10:10:05.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.267+0000 7f3fc6ffd640 1 -- 172.21.15.60:0/538641900 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3fb800e360 con 0x7f3fc8131d40 2023-08-24T10:10:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.267+0000 7f3fce59e640 1 -- 172.21.15.60:0/538641900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc8131d40 msgr2=0x7f3fc8134160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.267+0000 7f3fce59e640 1 --2- 172.21.15.60:0/538641900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc8131d40 0x7f3fc8134160 crc :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.267+0000 7f3fce59e640 1 -- 172.21.15.60:0/538641900 shutdown_connections 2023-08-24T10:10:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.267+0000 7f3fce59e640 1 --2- 172.21.15.60:0/538641900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc8131d40 0x7f3fc8134160 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.267+0000 7f3fce59e640 1 -- 172.21.15.60:0/538641900 >> 172.21.15.60:0/538641900 conn(0x7f3fc809e110 msgr2=0x7f3fc809e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:10:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.267+0000 7f3fce59e640 1 -- 172.21.15.60:0/538641900 shutdown_connections 2023-08-24T10:10:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.267+0000 7f3fce59e640 1 -- 172.21.15.60:0/538641900 wait complete. 2023-08-24T10:10:05.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.268+0000 7f3fce59e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:10:05.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.269+0000 7f3fce59e640 1 Processor -- start 2023-08-24T10:10:05.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.269+0000 7f3fce59e640 1 -- start start 2023-08-24T10:10:05.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.270+0000 7f3fce59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc806d300 0x7f3fc806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:05.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.270+0000 7f3fce59e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc806ff10 con 0x7f3fc806d300 2023-08-24T10:10:05.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.270+0000 7f3fc7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc806d300 0x7f3fc806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:05.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.270+0000 7f3fc7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc806d300 0x7f3fc806dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54378/0 (socket says 172.21.15.60:54378) 2023-08-24T10:10:05.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.270+0000 7f3fc7fff640 1 -- 172.21.15.60:0/1854594662 learned_addr learned my addr 172.21.15.60:0/1854594662 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:10:05.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.270+0000 7f3fc7fff640 1 -- 172.21.15.60:0/1854594662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc8071c10 con 0x7f3fc806d300 2023-08-24T10:10:05.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.270+0000 7f3fc7fff640 1 --2- 172.21.15.60:0/1854594662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc806d300 0x7f3fc806dc90 crc :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:05.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.270+0000 7f3fc4ff9640 1 -- 172.21.15.60:0/1854594662 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3fbc0059d0 con 0x7f3fc806d300 2023-08-24T10:10:05.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.270+0000 7f3fc4ff9640 1 -- 172.21.15.60:0/1854594662 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3fbc005b30 con 0x7f3fc806d300 2023-08-24T10:10:05.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.270+0000 7f3fce59e640 1 -- 172.21.15.60:0/1854594662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc8070660 con 0x7f3fc806d300 2023-08-24T10:10:05.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.270+0000 7f3fc4ff9640 1 -- 172.21.15.60:0/1854594662 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3fbc005dd0 con 0x7f3fc806d300 2023-08-24T10:10:05.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.270+0000 7f3fce59e640 1 -- 172.21.15.60:0/1854594662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc812b040 con 0x7f3fc806d300 2023-08-24T10:10:05.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.271+0000 7f3fc4ff9640 1 -- 172.21.15.60:0/1854594662 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f3fbc0070b0 con 0x7f3fc806d300 2023-08-24T10:10:05.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.271+0000 7f3fc4ff9640 1 --2- 172.21.15.60:0/1854594662 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3fb4037130 0x7f3fb4039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:05.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.271+0000 7f3fc4ff9640 1 -- 172.21.15.60:0/1854594662 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f3fbc0463b0 con 0x7f3fc806d300 2023-08-24T10:10:05.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.271+0000 7f3fce59e640 1 -- 172.21.15.60:0/1854594662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc8061800 con 0x7f3fc806d300 2023-08-24T10:10:05.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.272+0000 7f3fc77fe640 1 --2- 172.21.15.60:0/1854594662 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3fb4037130 0x7f3fb4039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:05.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.272+0000 7f3fc77fe640 1 --2- 172.21.15.60:0/1854594662 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3fb4037130 0x7f3fb4039640 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:05.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.276+0000 7f3fc4ff9640 1 -- 172.21.15.60:0/1854594662 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f3fbc00d720 con 0x7f3fc806d300 2023-08-24T10:10:05.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.448+0000 7f3fce59e640 1 -- 172.21.15.60:0/1854594662 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3fc81363a0 con 0x7f3fb4037130 2023-08-24T10:10:05.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.449+0000 7f3fc4ff9640 1 -- 172.21.15.60:0/1854594662 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f3fb8009c20 con 0x7f3fb4037130 2023-08-24T10:10:05.452 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:10:05.453 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:10:05.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.451+0000 7f3fce59e640 1 -- 172.21.15.60:0/1854594662 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3fb4037130 msgr2=0x7f3fb4039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:05.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.451+0000 7f3fce59e640 1 --2- 172.21.15.60:0/1854594662 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3fb4037130 0x7f3fb4039640 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.451+0000 7f3fce59e640 1 -- 172.21.15.60:0/1854594662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc806d300 msgr2=0x7f3fc806dc90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.451+0000 7f3fce59e640 1 --2- 172.21.15.60:0/1854594662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc806d300 0x7f3fc806dc90 crc :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.451+0000 7f3fce59e640 1 -- 172.21.15.60:0/1854594662 shutdown_connections 2023-08-24T10:10:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.451+0000 7f3fce59e640 1 --2- 172.21.15.60:0/1854594662 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f3fb4037130 0x7f3fb4039640 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:05.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.451+0000 7f3fce59e640 1 --2- 172.21.15.60:0/1854594662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc806d300 0x7f3fc806dc90 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:05.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.451+0000 7f3fce59e640 1 -- 172.21.15.60:0/1854594662 >> 172.21.15.60:0/1854594662 conn(0x7f3fc809e110 msgr2=0x7f3fc8097710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:10:05.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.451+0000 7f3fce59e640 1 -- 172.21.15.60:0/1854594662 shutdown_connections 2023-08-24T10:10:05.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.451+0000 7f3fce59e640 1 -- 172.21.15.60:0/1854594662 wait complete. 2023-08-24T10:10:05.464 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2023-08-24T10:10:03.553715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:10:05.465 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:10:05.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.570+0000 7f701091e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:10:05.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.574+0000 7f701091e640 1 Processor -- start 2023-08-24T10:10:05.576 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.574+0000 7f701091e640 1 -- start start 2023-08-24T10:10:05.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.575+0000 7f701091e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f700c070110 0x7f700c1356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:05.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.575+0000 7f701091e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700c096f10 con 0x7f700c071e40 2023-08-24T10:10:05.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.575+0000 7f701091e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700c0977f0 con 0x7f700c070110 2023-08-24T10:10:05.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.575+0000 7f700a59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f700c071e40 0x7f700c096800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49866/0 (socket says 172.21.15.60:49866) 2023-08-24T10:10:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.575+0000 7f700a59c640 1 -- 172.21.15.60:0/1532042629 learned_addr learned my addr 172.21.15.60:0/1532042629 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-08-24T10:10:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.575+0000 7f7003fff640 1 --2- 172.21.15.60:0/1532042629 >> v2:172.21.15.60:3300/0 conn(0x7f700c070110 0x7f700c1356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.575+0000 7f7003fff640 1 -- 172.21.15.60:0/1532042629 >> v1:172.21.15.60:6789/0 conn(0x7f700c071e40 legacy=0x7f700c096800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.575+0000 7f7003fff640 1 -- 172.21.15.60:0/1532042629 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c09a280 con 0x7f700c070110 2023-08-24T10:10:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.575+0000 7f7003fff640 1 --2- 172.21.15.60:0/1532042629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f700c070110 0x7f700c1356a0 crc :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b478264ba726f7 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.575+0000 7f7009d9b640 1 -- 172.21.15.60:0/1532042629 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6ff4009ce0 con 0x7f700c070110 2023-08-24T10:10:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.576+0000 7f7009d9b640 1 -- 172.21.15.60:0/1532042629 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f6ff4009e40 con 0x7f700c070110 2023-08-24T10:10:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.576+0000 7f7009d9b640 1 -- 172.21.15.60:0/1532042629 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6ff400e360 con 0x7f700c070110 2023-08-24T10:10:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.576+0000 7f701091e640 1 -- 172.21.15.60:0/1532042629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f700c070110 msgr2=0x7f700c1356a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.576+0000 7f701091e640 1 --2- 172.21.15.60:0/1532042629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f700c070110 0x7f700c1356a0 crc :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.576+0000 7f701091e640 1 -- 172.21.15.60:0/1532042629 shutdown_connections 2023-08-24T10:10:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.576+0000 7f701091e640 1 --2- 172.21.15.60:0/1532042629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f700c070110 0x7f700c1356a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.576+0000 7f701091e640 1 -- 172.21.15.60:0/1532042629 >> 172.21.15.60:0/1532042629 conn(0x7f700c09e110 msgr2=0x7f700c09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:10:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.576+0000 7f701091e640 1 -- 172.21.15.60:0/1532042629 shutdown_connections 2023-08-24T10:10:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.576+0000 7f701091e640 1 -- 172.21.15.60:0/1532042629 wait complete. 2023-08-24T10:10:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.577+0000 7f701091e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:10:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.578+0000 7f701091e640 1 Processor -- start 2023-08-24T10:10:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.578+0000 7f701091e640 1 -- start start 2023-08-24T10:10:05.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.578+0000 7f701091e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f700c070110 0x7f700c06cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.578+0000 7f701091e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c06e280 con 0x7f700c070110 2023-08-24T10:10:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.578+0000 7f700a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f700c070110 0x7f700c06cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.578+0000 7f700a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f700c070110 0x7f700c06cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54402/0 (socket says 172.21.15.60:54402) 2023-08-24T10:10:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.578+0000 7f700a59c640 1 -- 172.21.15.60:0/75709086 learned_addr learned my addr 172.21.15.60:0/75709086 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:10:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.579+0000 7f700a59c640 1 -- 172.21.15.60:0/75709086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c06e8f0 con 0x7f700c070110 2023-08-24T10:10:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.579+0000 7f700a59c640 1 --2- 172.21.15.60:0/75709086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f700c070110 0x7f700c06cef0 crc :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.579+0000 7f7002ffd640 1 -- 172.21.15.60:0/75709086 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6ffc0059d0 con 0x7f700c070110 2023-08-24T10:10:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.579+0000 7f7002ffd640 1 -- 172.21.15.60:0/75709086 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f6ffc005b30 con 0x7f700c070110 2023-08-24T10:10:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.579+0000 7f701091e640 1 -- 172.21.15.60:0/75709086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c125eb0 con 0x7f700c070110 2023-08-24T10:10:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.579+0000 7f7002ffd640 1 -- 172.21.15.60:0/75709086 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6ffc005dd0 con 0x7f700c070110 2023-08-24T10:10:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.579+0000 7f701091e640 1 -- 172.21.15.60:0/75709086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c06ee20 con 0x7f700c070110 2023-08-24T10:10:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.580+0000 7f7002ffd640 1 -- 172.21.15.60:0/75709086 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f6ffc0070b0 con 0x7f700c070110 2023-08-24T10:10:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.580+0000 7f701091e640 1 -- 172.21.15.60:0/75709086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700c097a50 con 0x7f700c070110 2023-08-24T10:10:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.580+0000 7f7002ffd640 1 --2- 172.21.15.60:0/75709086 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6fec037180 0x7f6fec039690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.580+0000 7f7002ffd640 1 -- 172.21.15.60:0/75709086 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f6ffc046380 con 0x7f700c070110 2023-08-24T10:10:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.580+0000 7f7003fff640 1 --2- 172.21.15.60:0/75709086 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6fec037180 0x7f6fec039690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.581+0000 7f7003fff640 1 --2- 172.21.15.60:0/75709086 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6fec037180 0x7f6fec039690 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.585+0000 7f7002ffd640 1 -- 172.21.15.60:0/75709086 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f6ffc00d330 con 0x7f700c070110 2023-08-24T10:10:05.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.755+0000 7f701091e640 1 -- 172.21.15.60:0/75709086 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f700c070970 con 0x7f6fec037180 2023-08-24T10:10:05.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.757+0000 7f7002ffd640 1 -- 172.21.15.60:0/75709086 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f6ff4012490 con 0x7f6fec037180 2023-08-24T10:10:05.759 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:10:05.759 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:10:05.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.758+0000 7f701091e640 1 -- 172.21.15.60:0/75709086 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6fec037180 msgr2=0x7f6fec039690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:05.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.758+0000 7f701091e640 1 --2- 172.21.15.60:0/75709086 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6fec037180 0x7f6fec039690 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:05.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.759+0000 7f701091e640 1 -- 172.21.15.60:0/75709086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f700c070110 msgr2=0x7f700c06cef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:05.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.759+0000 7f701091e640 1 --2- 172.21.15.60:0/75709086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f700c070110 0x7f700c06cef0 crc :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:05.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.759+0000 7f701091e640 1 -- 172.21.15.60:0/75709086 shutdown_connections 2023-08-24T10:10:05.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.759+0000 7f701091e640 1 --2- 172.21.15.60:0/75709086 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f6fec037180 0x7f6fec039690 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:05.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.759+0000 7f701091e640 1 --2- 172.21.15.60:0/75709086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f700c070110 0x7f700c06cef0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:05.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.759+0000 7f701091e640 1 -- 172.21.15.60:0/75709086 >> 172.21.15.60:0/75709086 conn(0x7f700c09e110 msgr2=0x7f700c098490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:10:05.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.759+0000 7f701091e640 1 -- 172.21.15.60:0/75709086 shutdown_connections 2023-08-24T10:10:05.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.759+0000 7f701091e640 1 -- 172.21.15.60:0/75709086 wait complete. 2023-08-24T10:10:05.771 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2023-08-24T10:10:05.554231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:10:05.772 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2023-08-24T10:10:05.772 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:10:05.773 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2023-08-24T10:10:05.773 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:10:05.773 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2023-08-24T10:10:05.773 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:10:05.774 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2023-08-24T10:10:05.774 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:10:05.774 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2023-08-24T10:10:05.774 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:10:05.775 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2023-08-24T10:10:05.775 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:10:05.775 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2023-08-24T10:10:05.776 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:10:05.776 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2023-08-24T10:10:05.777 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:05:00.616547+0000', 'last_change': '2023-08-24T10:05:00.616547+0000', 'last_active': '2023-08-24T10:05:00.616547+0000', 'last_peered': '2023-08-24T10:05:00.616547+0000', 'last_clean': '2023-08-24T10:05:00.616547+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:05:00.616547+0000', 'last_undegraded': '2023-08-24T10:05:00.616547+0000', 'last_fullsized': '2023-08-24T10:05:00.616547+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'last_clean_scrub_stamp': '2023-08-24T10:05:00.616547+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:10:05.777 INFO:tasks.ceph.ceph_manager.ceph:PG 1.0 is not active+clean 2023-08-24T10:10:05.777 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-08-24T10:04:57.603921+0000', 'last_change': '2023-08-24T10:04:57.603921+0000', 'last_active': '2023-08-24T10:04:57.603921+0000', 'last_peered': '2023-08-24T10:04:57.603921+0000', 'last_clean': '2023-08-24T10:04:57.603921+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-24T10:04:57.603921+0000', 'last_undegraded': '2023-08-24T10:04:57.603921+0000', 'last_fullsized': '2023-08-24T10:04:57.603921+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'last_clean_scrub_stamp': '2023-08-24T10:04:57.603921+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-08-24T10:10:05.778 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-24T10:10:05.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.884+0000 7f216dcec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:10:05.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.886+0000 7f216dcec640 1 Processor -- start 2023-08-24T10:10:05.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.887+0000 7f216dcec640 1 -- start start 2023-08-24T10:10:05.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.887+0000 7f216dcec640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f216812f610 0x7f216812fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:05.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.887+0000 7f216dcec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216809d750 con 0x7f21681326f0 2023-08-24T10:10:05.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.887+0000 7f216dcec640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216809d980 con 0x7f216812f610 2023-08-24T10:10:05.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.887+0000 7f21677fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f216812f610 0x7f216812fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:05.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.887+0000 7f21677fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f216812f610 0x7f216812fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54408/0 (socket says 172.21.15.60:54408) 2023-08-24T10:10:05.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.887+0000 7f21677fe640 1 -- 172.21.15.60:0/1443378865 learned_addr learned my addr 172.21.15.60:0/1443378865 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:10:05.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.887+0000 7f2166ffd640 1 --1- 172.21.15.60:0/1443378865 >> v1:172.21.15.60:6789/0 conn(0x7f21681326f0 0x7f2168132b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49876/0 (socket says 172.21.15.60:49876) 2023-08-24T10:10:05.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.888+0000 7f21677fe640 1 -- 172.21.15.60:0/1443378865 >> v1:172.21.15.60:6789/0 conn(0x7f21681326f0 legacy=0x7f2168132b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:05.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.888+0000 7f21677fe640 1 -- 172.21.15.60:0/1443378865 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216809bde0 con 0x7f216812f610 2023-08-24T10:10:05.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.888+0000 7f21677fe640 1 --2- 172.21.15.60:0/1443378865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f216812f610 0x7f216812fa20 crc :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c322e8cb453a8454 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:05.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.888+0000 7f21667fc640 1 -- 172.21.15.60:0/1443378865 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2158009dc0 con 0x7f216812f610 2023-08-24T10:10:05.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.888+0000 7f21667fc640 1 -- 172.21.15.60:0/1443378865 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f215800e230 con 0x7f216812f610 2023-08-24T10:10:05.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.888+0000 7f21667fc640 1 -- 172.21.15.60:0/1443378865 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f215800e4d0 con 0x7f216812f610 2023-08-24T10:10:05.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.889+0000 7f216dcec640 1 -- 172.21.15.60:0/1443378865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f216812f610 msgr2=0x7f216812fa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:05.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.889+0000 7f216dcec640 1 --2- 172.21.15.60:0/1443378865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f216812f610 0x7f216812fa20 crc :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:05.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.889+0000 7f216dcec640 1 -- 172.21.15.60:0/1443378865 shutdown_connections 2023-08-24T10:10:05.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.889+0000 7f216dcec640 1 --2- 172.21.15.60:0/1443378865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f216812f610 0x7f216812fa20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:05.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.889+0000 7f216dcec640 1 -- 172.21.15.60:0/1443378865 >> 172.21.15.60:0/1443378865 conn(0x7f21680a2500 msgr2=0x7f21680a2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:10:05.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.889+0000 7f216dcec640 1 -- 172.21.15.60:0/1443378865 shutdown_connections 2023-08-24T10:10:05.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.889+0000 7f216dcec640 1 -- 172.21.15.60:0/1443378865 wait complete. 2023-08-24T10:10:05.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.889+0000 7f216dcec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:10:05.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.890+0000 7f216dcec640 1 Processor -- start 2023-08-24T10:10:05.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.890+0000 7f216dcec640 1 -- start start 2023-08-24T10:10:05.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.891+0000 7f216dcec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f216812f610 0x7f216809fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:05.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.891+0000 7f216dcec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216812cc60 con 0x7f216812f610 2023-08-24T10:10:05.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.891+0000 7f21677fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f216812f610 0x7f216809fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:05.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.891+0000 7f21677fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f216812f610 0x7f216809fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54420/0 (socket says 172.21.15.60:54420) 2023-08-24T10:10:05.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.891+0000 7f21677fe640 1 -- 172.21.15.60:0/1202160825 learned_addr learned my addr 172.21.15.60:0/1202160825 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:10:05.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.891+0000 7f21677fe640 1 -- 172.21.15.60:0/1202160825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216806a1f0 con 0x7f216812f610 2023-08-24T10:10:05.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.891+0000 7f21677fe640 1 --2- 172.21.15.60:0/1202160825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f216812f610 0x7f216809fca0 crc :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:05.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.891+0000 7f214ffff640 1 -- 172.21.15.60:0/1202160825 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2158014540 con 0x7f216812f610 2023-08-24T10:10:05.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.891+0000 7f214ffff640 1 -- 172.21.15.60:0/1202160825 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f21580146a0 con 0x7f216812f610 2023-08-24T10:10:05.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.891+0000 7f214ffff640 1 -- 172.21.15.60:0/1202160825 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2158014940 con 0x7f216812f610 2023-08-24T10:10:05.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.891+0000 7f216dcec640 1 -- 172.21.15.60:0/1202160825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216809f7e0 con 0x7f216812f610 2023-08-24T10:10:05.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.891+0000 7f216dcec640 1 -- 172.21.15.60:0/1202160825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216809f4d0 con 0x7f216812f610 2023-08-24T10:10:05.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.892+0000 7f214ffff640 1 -- 172.21.15.60:0/1202160825 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f2158014af0 con 0x7f216812f610 2023-08-24T10:10:05.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.892+0000 7f216dcec640 1 -- 172.21.15.60:0/1202160825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2168061800 con 0x7f216812f610 2023-08-24T10:10:05.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.892+0000 7f214ffff640 1 --2- 172.21.15.60:0/1202160825 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2154037130 0x7f2154039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:05.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.892+0000 7f214ffff640 1 -- 172.21.15.60:0/1202160825 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f2158019040 con 0x7f216812f610 2023-08-24T10:10:05.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.893+0000 7f2166ffd640 1 --2- 172.21.15.60:0/1202160825 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2154037130 0x7f2154039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:05.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.893+0000 7f2166ffd640 1 --2- 172.21.15.60:0/1202160825 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2154037130 0x7f2154039640 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:05.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:05.897+0000 7f214ffff640 1 -- 172.21.15.60:0/1202160825 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f215800fb00 con 0x7f216812f610 2023-08-24T10:10:06.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.143+0000 7f216dcec640 1 -- 172.21.15.60:0/1202160825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f216806a1f0 con 0x7f216812f610 2023-08-24T10:10:06.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.143+0000 7f214ffff640 1 -- 172.21.15.60:0/1202160825 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+950 (crc 0 0 0) 0x7f216806a1f0 con 0x7f216812f610 2023-08-24T10:10:06.147 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:10:06.147 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"bf8addf4-f373-46ca-83a9-ccdf9773802d","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 9 pgs inactive","count":9},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":316,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":14,"num_osds":4,"num_up_osds":4,"osd_up_since":1692871495,"num_in_osds":4,"osd_in_since":1692871491,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":9}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":0,"bytes_avail":0,"bytes_total":0,"unknown_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2023-08-24T10:05:55.506913+0000","services":{}},"progress_events":{}} 2023-08-24T10:10:06.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.145+0000 7f216dcec640 1 -- 172.21.15.60:0/1202160825 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2154037130 msgr2=0x7f2154039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:06.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.145+0000 7f216dcec640 1 --2- 172.21.15.60:0/1202160825 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2154037130 0x7f2154039640 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:06.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.145+0000 7f216dcec640 1 -- 172.21.15.60:0/1202160825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f216812f610 msgr2=0x7f216809fca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:06.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.145+0000 7f216dcec640 1 --2- 172.21.15.60:0/1202160825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f216812f610 0x7f216809fca0 crc :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:06.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.145+0000 7f216dcec640 1 -- 172.21.15.60:0/1202160825 shutdown_connections 2023-08-24T10:10:06.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.145+0000 7f216dcec640 1 --2- 172.21.15.60:0/1202160825 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f2154037130 0x7f2154039640 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:06.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.145+0000 7f216dcec640 1 --2- 172.21.15.60:0/1202160825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f216812f610 0x7f216809fca0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:06.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.145+0000 7f216dcec640 1 -- 172.21.15.60:0/1202160825 >> 172.21.15.60:0/1202160825 conn(0x7f21680a2500 msgr2=0x7f21681ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:10:06.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.145+0000 7f216dcec640 1 -- 172.21.15.60:0/1202160825 shutdown_connections 2023-08-24T10:10:06.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.145+0000 7f216dcec640 1 -- 172.21.15.60:0/1202160825 wait complete. 2023-08-24T10:10:06.158 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-08-24T10:10:06.158 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:10:06.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.264+0000 7f0a42355640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:10:06.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.268+0000 7f0a42355640 1 Processor -- start 2023-08-24T10:10:06.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.268+0000 7f0a42355640 1 -- start start 2023-08-24T10:10:06.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.269+0000 7f0a42355640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0a3c070110 0x7f0a3c1356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:06.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.269+0000 7f0a42355640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3c096f10 con 0x7f0a3c071e40 2023-08-24T10:10:06.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.269+0000 7f0a42355640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3c0977f0 con 0x7f0a3c070110 2023-08-24T10:10:06.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.269+0000 7f0a3b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0a3c070110 0x7f0a3c1356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:06.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.269+0000 7f0a3b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0a3c070110 0x7f0a3c1356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54422/0 (socket says 172.21.15.60:54422) 2023-08-24T10:10:06.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.269+0000 7f0a3b7fe640 1 -- 172.21.15.60:0/719397046 learned_addr learned my addr 172.21.15.60:0/719397046 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:10:06.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.269+0000 7f0a3bfff640 1 --1- 172.21.15.60:0/719397046 >> v1:172.21.15.60:6789/0 conn(0x7f0a3c071e40 0x7f0a3c096800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49884/0 (socket says 172.21.15.60:49884) 2023-08-24T10:10:06.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.269+0000 7f0a3b7fe640 1 -- 172.21.15.60:0/719397046 >> v1:172.21.15.60:6789/0 conn(0x7f0a3c071e40 legacy=0x7f0a3c096800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:06.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.269+0000 7f0a3b7fe640 1 -- 172.21.15.60:0/719397046 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c09a280 con 0x7f0a3c070110 2023-08-24T10:10:06.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.269+0000 7f0a3b7fe640 1 --2- 172.21.15.60:0/719397046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a3c070110 0x7f0a3c1356a0 crc :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd711537a090a6f server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:06.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.269+0000 7f0a3affd640 1 -- 172.21.15.60:0/719397046 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0a2c009dd0 con 0x7f0a3c070110 2023-08-24T10:10:06.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.269+0000 7f0a3affd640 1 -- 172.21.15.60:0/719397046 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0a2c00e230 con 0x7f0a3c070110 2023-08-24T10:10:06.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.269+0000 7f0a3affd640 1 -- 172.21.15.60:0/719397046 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0a2c00e4d0 con 0x7f0a3c070110 2023-08-24T10:10:06.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.270+0000 7f0a42355640 1 -- 172.21.15.60:0/719397046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a3c070110 msgr2=0x7f0a3c1356a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:06.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.270+0000 7f0a42355640 1 --2- 172.21.15.60:0/719397046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a3c070110 0x7f0a3c1356a0 crc :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:06.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.270+0000 7f0a42355640 1 -- 172.21.15.60:0/719397046 shutdown_connections 2023-08-24T10:10:06.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.270+0000 7f0a42355640 1 --2- 172.21.15.60:0/719397046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a3c070110 0x7f0a3c1356a0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:06.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.270+0000 7f0a42355640 1 -- 172.21.15.60:0/719397046 >> 172.21.15.60:0/719397046 conn(0x7f0a3c09e110 msgr2=0x7f0a3c09e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:10:06.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.270+0000 7f0a42355640 1 -- 172.21.15.60:0/719397046 shutdown_connections 2023-08-24T10:10:06.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.270+0000 7f0a42355640 1 -- 172.21.15.60:0/719397046 wait complete. 2023-08-24T10:10:06.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.270+0000 7f0a42355640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:10:06.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.271+0000 7f0a42355640 1 Processor -- start 2023-08-24T10:10:06.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.271+0000 7f0a42355640 1 -- start start 2023-08-24T10:10:06.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.272+0000 7f0a42355640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a3c070110 0x7f0a3c093460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:06.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.272+0000 7f0a42355640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c08ff10 con 0x7f0a3c070110 2023-08-24T10:10:06.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.272+0000 7f0a3bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a3c070110 0x7f0a3c093460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:06.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.272+0000 7f0a3bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a3c070110 0x7f0a3c093460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54430/0 (socket says 172.21.15.60:54430) 2023-08-24T10:10:06.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.272+0000 7f0a3bfff640 1 -- 172.21.15.60:0/2580402423 learned_addr learned my addr 172.21.15.60:0/2580402423 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-08-24T10:10:06.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.272+0000 7f0a3bfff640 1 -- 172.21.15.60:0/2580402423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c090680 con 0x7f0a3c070110 2023-08-24T10:10:06.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.273+0000 7f0a3bfff640 1 --2- 172.21.15.60:0/2580402423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a3c070110 0x7f0a3c093460 crc :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:06.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.273+0000 7f0a38ff9640 1 -- 172.21.15.60:0/2580402423 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0a240071e0 con 0x7f0a3c070110 2023-08-24T10:10:06.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.273+0000 7f0a38ff9640 1 -- 172.21.15.60:0/2580402423 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0a24007340 con 0x7f0a3c070110 2023-08-24T10:10:06.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.273+0000 7f0a38ff9640 1 -- 172.21.15.60:0/2580402423 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0a240075e0 con 0x7f0a3c070110 2023-08-24T10:10:06.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.273+0000 7f0a42355640 1 -- 172.21.15.60:0/2580402423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3c092e80 con 0x7f0a3c070110 2023-08-24T10:10:06.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.273+0000 7f0a42355640 1 -- 172.21.15.60:0/2580402423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3c092b70 con 0x7f0a3c070110 2023-08-24T10:10:06.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.274+0000 7f0a38ff9640 1 -- 172.21.15.60:0/2580402423 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 44411+0+0 (crc 0 0 0) 0x7f0a240057e0 con 0x7f0a3c070110 2023-08-24T10:10:06.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.274+0000 7f0a42355640 1 -- 172.21.15.60:0/2580402423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3c061800 con 0x7f0a3c070110 2023-08-24T10:10:06.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.274+0000 7f0a38ff9640 1 --2- 172.21.15.60:0/2580402423 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0a1c037130 0x7f0a1c039640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:06.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.274+0000 7f0a38ff9640 1 -- 172.21.15.60:0/2580402423 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f0a240463b0 con 0x7f0a3c070110 2023-08-24T10:10:06.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.274+0000 7f0a3b7fe640 1 --2- 172.21.15.60:0/2580402423 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0a1c037130 0x7f0a1c039640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-24T10:10:06.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.275+0000 7f0a3b7fe640 1 --2- 172.21.15.60:0/2580402423 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0a1c037130 0x7f0a1c039640 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-24T10:10:06.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.278+0000 7f0a38ff9640 1 -- 172.21.15.60:0/2580402423 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (crc 0 0 0) 0x7f0a2400d330 con 0x7f0a3c070110 2023-08-24T10:10:06.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.451+0000 7f0a42355640 1 -- 172.21.15.60:0/2580402423 --> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0a3c0932f0 con 0x7f0a1c037130 2023-08-24T10:10:06.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.453+0000 7f0a38ff9640 1 -- 172.21.15.60:0/2580402423 <== mgr.4099 v2:172.21.15.135:6800/3444395587 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+26402 (crc 0 0 0) 0x7f0a3c0932f0 con 0x7f0a1c037130 2023-08-24T10:10:06.456 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-08-24T10:10:06.457 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-08-24T10:10:06.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.454+0000 7f0a42355640 1 -- 172.21.15.60:0/2580402423 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0a1c037130 msgr2=0x7f0a1c039640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:06.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.454+0000 7f0a42355640 1 --2- 172.21.15.60:0/2580402423 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0a1c037130 0x7f0a1c039640 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:06.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.454+0000 7f0a42355640 1 -- 172.21.15.60:0/2580402423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a3c070110 msgr2=0x7f0a3c093460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-24T10:10:06.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.454+0000 7f0a42355640 1 --2- 172.21.15.60:0/2580402423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a3c070110 0x7f0a3c093460 crc :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:06.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.454+0000 7f0a42355640 1 -- 172.21.15.60:0/2580402423 shutdown_connections 2023-08-24T10:10:06.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.454+0000 7f0a42355640 1 --2- 172.21.15.60:0/2580402423 >> [v2:172.21.15.135:6800/3444395587,v1:172.21.15.135:6801/3444395587] conn(0x7f0a1c037130 0x7f0a1c039640 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:06.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.454+0000 7f0a42355640 1 --2- 172.21.15.60:0/2580402423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a3c070110 0x7f0a3c093460 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:06.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.454+0000 7f0a42355640 1 -- 172.21.15.60:0/2580402423 >> 172.21.15.60:0/2580402423 conn(0x7f0a3c09e110 msgr2=0x7f0a3c098490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-24T10:10:06.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.454+0000 7f0a42355640 1 -- 172.21.15.60:0/2580402423 shutdown_connections 2023-08-24T10:10:06.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:06.454+0000 7f0a42355640 1 -- 172.21.15.60:0/2580402423 wait complete. 2023-08-24T10:10:06.466 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2023-08-24T10:10:05.554231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:05:00.616547+0000","last_change":"2023-08-24T10:05:00.616547+0000","last_active":"2023-08-24T10:05:00.616547+0000","last_peered":"2023-08-24T10:05:00.616547+0000","last_clean":"2023-08-24T10:05:00.616547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:05:00.616547+0000","last_undegraded":"2023-08-24T10:05:00.616547+0000","last_fullsized":"2023-08-24T10:05:00.616547+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:05:00.616547+0000","last_clean_scrub_stamp":"2023-08-24T10:05:00.616547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-08-24T10:04:57.603921+0000","last_change":"2023-08-24T10:04:57.603921+0000","last_active":"2023-08-24T10:04:57.603921+0000","last_peered":"2023-08-24T10:04:57.603921+0000","last_clean":"2023-08-24T10:04:57.603921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-24T10:04:57.603921+0000","last_undegraded":"2023-08-24T10:04:57.603921+0000","last_fullsized":"2023-08-24T10:04:57.603921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-24T10:04:57.603921+0000","last_clean_scrub_stamp":"2023-08-24T10:04:57.603921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[],"pool_statfs":[]}} 2023-08-24T10:10:09.277 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~302s 2023-08-24T10:10:09.277 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~302s 2023-08-24T10:10:09.278 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~302s 2023-08-24T10:10:09.278 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~302s 2023-08-24T10:10:09.278 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2023-08-24T10:10:09.468 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:10:09.480 INFO:tasks.ceph.mon.a:Sent signal 15 2023-08-24T10:10:09.481 INFO:tasks.ceph.mgr.x:Sent signal 15 2023-08-24T10:10:09.482 INFO:tasks.ceph.mgr.x.smithi135.stderr:2023-08-24T10:10:09.481+0000 7fe1033f7640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mgr -f --cluster ceph -i x (PID: 82599) UID: 0 2023-08-24T10:10:09.512 INFO:tasks.ceph.mon.a.smithi060.stderr:2023-08-24T10:10:09.510+0000 7fa56f5fc640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 82991) UID: 0 2023-08-24T10:10:09.512 INFO:tasks.ceph.mon.a.smithi060.stderr:2023-08-24T10:10:09.510+0000 7fa56f5fc640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-08-24T10:10:09.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:09.584+0000 7f21dd19e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:10:09.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:09.588+0000 7f21dd19e640 1 Processor -- start 2023-08-24T10:10:09.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:09.588+0000 7f21dd19e640 1 -- start start 2023-08-24T10:10:09.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:09.589+0000 7f21dd19e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 0x7f21d8137da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:09.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:09.589+0000 7f21dd19e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21d8132b50 2023-08-24T10:10:09.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:09.589+0000 7f21dd19e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21d8135980 2023-08-24T10:10:09.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:09.589+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21d8132f60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:09.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:09.589+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 msgr2=0x7f21d8137da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:09.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:09.589+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 0x7f21d8137da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:10:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:09.790+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21d8132f60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:09.790+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 msgr2=0x7f21d8137da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:09.790+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 0x7f21d8137da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:10:10.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:10.191+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 msgr2=0x7f21d8137da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:10.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:10.191+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21d8132f60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:10.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:10.191+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 0x7f21d8137da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:10:10.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:10.992+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21d8132f60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:10.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:10.992+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 msgr2=0x7f21d8137da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:10.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:10.992+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 0x7f21d8137da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:10:12.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:12.590+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 msgr2=0x7f21d8137da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:12.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:12.590+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 0x7f21d8137da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:12.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:12.590+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21d8132f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:12.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:12.590+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 0x7f21b8008e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:12.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:12.590+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21d8132b50 2023-08-24T10:10:12.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:12.590+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21d8135980 2023-08-24T10:10:12.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:12.590+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:12.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:12.590+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 msgr2=0x7f21b8008e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:12.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:12.590+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 0x7f21b8008e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:10:12.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:12.791+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:12.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:12.791+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 msgr2=0x7f21b8008e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:12.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:12.791+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 0x7f21b8008e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:10:13.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:13.192+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:13.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:13.192+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 msgr2=0x7f21b8008e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:13.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:13.192+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 0x7f21b8008e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:10:13.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:13.993+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:13.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:13.993+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 msgr2=0x7f21b8008e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:13.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:13.993+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 0x7f21b8008e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:10:15.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:15.591+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 msgr2=0x7f21b8008e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:15.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:15.591+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21d8135980 0x7f21b8008e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:15.591+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:15.591+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:15.591+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21d8132b50 2023-08-24T10:10:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:15.591+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b80094c0 2023-08-24T10:10:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:15.591+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:15.591+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:15.591+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:10:15.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:15.792+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:15.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:15.792+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:15.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:15.792+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:10:16.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:16.193+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:16.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:16.193+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:16.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:16.193+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:10:16.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:16.994+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:16.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:16.994+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:16.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:16.994+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:10:18.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:18.591+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:18.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:18.591+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:18.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:18.591+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:18.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:18.591+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:18.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:18.591+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21d8132b50 2023-08-24T10:10:18.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:18.591+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b80094c0 2023-08-24T10:10:18.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:18.591+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:18.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:18.591+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:18.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:18.591+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:10:18.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:18.792+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:18.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:18.792+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:18.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:18.792+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:10:19.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:19.193+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:19.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:19.193+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:19.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:19.193+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:10:19.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:19.994+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:19.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:19.994+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:19.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:19.994+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:10:21.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:21.591+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:21.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:21.591+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:21.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:21.591+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:21.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:21.591+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:21.591+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21d8132b50 2023-08-24T10:10:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:21.591+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b80094c0 2023-08-24T10:10:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:21.591+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:21.591+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:21.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:21.591+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:10:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:21.792+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:21.792+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:21.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:21.792+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:10:22.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:22.193+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:22.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:22.193+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:22.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:22.193+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:10:22.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:22.994+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:22.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:22.994+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:22.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:22.994+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:10:24.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:24.592+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:24.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:24.592+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:24.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:24.592+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:24.592+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:24.592+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21d8132b50 2023-08-24T10:10:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:24.592+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b80094c0 2023-08-24T10:10:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:24.592+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:24.592+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:24.592+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:10:24.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:24.793+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:24.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:24.793+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:24.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:24.793+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:10:25.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:25.194+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:25.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:25.194+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:25.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:25.194+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:10:25.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:25.995+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:25.995+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:25.995+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:10:27.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:27.592+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:27.592+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:27.592+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21d8132b50 legacy=0x7f21b8005d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:27.592+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:27.592+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21b8009e50 2023-08-24T10:10:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:27.592+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b80094c0 2023-08-24T10:10:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:27.592+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:27.592+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8009e50 legacy=0x7f21b8006450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:27.592+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:10:27.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:27.793+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8009e50 legacy=0x7f21b8006450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:27.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:27.793+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:27.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:27.793+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:10:28.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:28.194+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8009e50 legacy=0x7f21b8006450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:28.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:28.194+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:28.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:28.194+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:10:28.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:28.995+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8009e50 legacy=0x7f21b8006450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:28.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:28.995+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:28.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:28.995+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:10:30.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:30.592+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:30.592+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:30.592+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8009e50 legacy=0x7f21b8006450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:30.592+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:30.592+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21b8009e50 2023-08-24T10:10:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:30.592+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b80094c0 2023-08-24T10:10:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:30.592+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:30.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:30.592+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8009e50 legacy=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:30.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:30.592+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:10:30.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:30.793+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:30.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:30.793+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8009e50 legacy=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:30.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:30.793+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:10:31.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:31.194+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:31.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:31.194+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8009e50 legacy=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:31.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:31.194+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:10:31.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:31.995+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:31.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:31.995+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8009e50 legacy=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:31.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:31.995+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:10:33.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:33.593+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:33.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:33.593+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:33.593+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8009e50 legacy=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:33.593+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:33.593+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21b8004cc0 2023-08-24T10:10:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:33.593+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b80094c0 2023-08-24T10:10:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:33.593+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b80050d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:33.593+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:33.593+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:10:33.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:33.794+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b80050d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:33.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:33.794+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:33.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:33.794+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:10:34.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:34.195+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:34.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:34.195+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b80050d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:34.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:34.195+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:10:34.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:34.996+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:34.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:34.996+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b80050d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:34.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:34.996+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:10:36.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:36.593+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 msgr2=0x7f21b80098d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:36.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:36.593+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80094c0 0x7f21b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:36.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:36.593+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b80050d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:36.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:36.593+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:36.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:36.593+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21b8004cc0 2023-08-24T10:10:36.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:36.593+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b80067f0 2023-08-24T10:10:36.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:36.593+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:36.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:36.593+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b8003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:36.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:36.593+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:10:36.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:36.794+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b8003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:36.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:36.794+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:36.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:36.794+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:10:37.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:37.195+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b8003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:37.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:37.195+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:37.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:37.195+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:10:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:37.996+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:37.996+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b8003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:37.996+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:10:39.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:39.593+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:39.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:39.593+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:39.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:39.593+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b8003ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:39.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:39.593+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:39.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:39.593+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21b8004cc0 2023-08-24T10:10:39.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:39.593+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b80067f0 2023-08-24T10:10:39.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:39.593+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:39.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:39.593+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b8003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:39.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:39.593+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:10:39.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:39.794+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b8003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:39.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:39.794+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:39.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:39.794+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:10:40.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:40.195+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b8003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:40.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:40.195+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:40.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:40.195+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:10:40.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:40.996+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b8003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:40.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:40.996+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:40.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:40.996+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:10:42.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:42.594+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:42.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:42.594+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:42.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:42.594+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b8003ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:42.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:42.594+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:42.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:42.594+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21b8004cc0 2023-08-24T10:10:42.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:42.594+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b80067f0 2023-08-24T10:10:42.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:42.594+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:42.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:42.594+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b8003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:42.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:42.594+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:10:42.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:42.795+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b8003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:42.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:42.795+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:42.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:42.795+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:10:43.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:43.196+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b8003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:43.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:43.196+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:43.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:43.196+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:10:43.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:43.997+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:43.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:43.997+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b8003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:43.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:43.997+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:10:45.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:45.594+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:45.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:45.594+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:45.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:45.594+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8004cc0 legacy=0x7f21b8003ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:45.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:45.594+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:45.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:45.594+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21b80098d0 2023-08-24T10:10:45.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:45.594+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b80067f0 2023-08-24T10:10:45.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:45.594+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:45.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:45.594+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80098d0 legacy=0x7f21b80060e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:45.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:45.594+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:10:45.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:45.795+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80098d0 legacy=0x7f21b80060e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:45.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:45.795+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:45.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:45.795+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:10:46.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:46.196+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:46.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:46.196+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80098d0 legacy=0x7f21b80060e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:46.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:46.196+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:10:46.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:46.997+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80098d0 legacy=0x7f21b80060e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:46.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:46.997+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:46.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:46.997+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:10:48.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:48.594+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:48.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:48.594+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:48.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:48.594+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80098d0 legacy=0x7f21b80060e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:48.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:48.594+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:48.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:48.594+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21b80050d0 2023-08-24T10:10:48.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:48.594+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b80067f0 2023-08-24T10:10:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:48.594+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:48.594+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:48.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:48.594+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:10:48.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:48.795+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:48.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:48.795+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:48.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:48.795+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:10:49.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:49.196+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:49.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:49.196+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:49.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:49.196+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:10:49.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:49.997+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:49.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:49.997+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:49.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:49.997+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:10:51.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:51.595+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:51.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:51.595+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80067f0 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:51.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:51.595+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:51.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:51.595+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 0x7f21b8006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:51.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:51.595+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21b80050d0 2023-08-24T10:10:51.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:51.595+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b8005bf0 2023-08-24T10:10:51.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:51.595+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 msgr2=0x7f21b8006000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:51.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:51.595+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:51.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:51.595+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 0x7f21b8006000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:10:51.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:51.796+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:51.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:51.796+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 msgr2=0x7f21b8006000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:51.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:51.796+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 0x7f21b8006000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:10:52.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:52.197+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:52.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:52.197+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 msgr2=0x7f21b8006000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:52.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:52.197+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 0x7f21b8006000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:10:52.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:52.998+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:52.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:52.998+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 msgr2=0x7f21b8006000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:52.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:52.998+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 0x7f21b8006000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:10:54.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:54.595+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 msgr2=0x7f21b8006000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:54.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:54.595+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 0x7f21b8006000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:54.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:54.595+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:54.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:54.595+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 0x7f21b8006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:54.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:54.595+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21b80045b0 2023-08-24T10:10:54.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:54.595+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b8005bf0 2023-08-24T10:10:54.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:54.595+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b800e6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:54.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:54.595+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 msgr2=0x7f21b8006000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:54.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:54.595+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 0x7f21b8006000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:10:54.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:54.796+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 msgr2=0x7f21b8006000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:54.796+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b800e6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:54.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:54.796+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 0x7f21b8006000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:10:55.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:55.197+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 msgr2=0x7f21b8006000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:55.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:55.197+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b800e6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:55.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:55.197+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 0x7f21b8006000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:10:55.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:55.998+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 msgr2=0x7f21b8006000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:55.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:55.998+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b800e6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:55.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:55.998+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 0x7f21b8006000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:10:57.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:57.595+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 msgr2=0x7f21b8006000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:57.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:57.595+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005bf0 0x7f21b8006000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:10:57.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:57.595+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b800e6a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:10:57.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:57.595+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:10:57.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:57.595+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21b80045b0 2023-08-24T10:10:57.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:57.595+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b80050d0 2023-08-24T10:10:57.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:57.595+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:57.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:57.595+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:57.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:57.595+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:10:57.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:57.796+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:57.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:57.796+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:57.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:57.796+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:10:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:58.197+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:58.197+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:58.197+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:10:58.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:58.998+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:10:58.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:58.998+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:10:58.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:10:58.998+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:00.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:00.596+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:00.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:00.596+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:00.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:00.596+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:00.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:00.596+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:00.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:00.596+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21b80045b0 2023-08-24T10:11:00.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:00.596+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b80050d0 2023-08-24T10:11:00.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:00.596+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:00.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:00.596+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:00.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:00.596+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:00.797+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:00.797+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:00.797+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:01.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:01.198+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:01.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:01.198+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:01.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:01.198+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:02.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:01.999+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:02.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:01.999+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:02.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:01.999+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:03.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:03.596+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:03.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:03.596+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:03.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:03.596+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:03.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:03.596+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:03.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:03.596+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21b80045b0 2023-08-24T10:11:03.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:03.596+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b80050d0 2023-08-24T10:11:03.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:03.596+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:03.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:03.596+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:03.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:03.596+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:03.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:03.797+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:03.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:03.797+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:03.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:03.797+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:04.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:04.198+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:04.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:04.198+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:04.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:04.198+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:04.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:04.999+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:05.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:04.999+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:05.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:04.999+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:06.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:06.597+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:06.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:06.597+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:06.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:06.597+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:06.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:06.597+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:06.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:06.597+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21b80045b0 2023-08-24T10:11:06.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:06.597+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b80050d0 2023-08-24T10:11:06.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:06.597+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:06.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:06.597+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:06.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:06.597+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:06.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:06.798+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:06.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:06.798+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:06.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:06.798+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:07.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:07.199+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:07.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:07.199+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:07.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:07.199+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:08.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:08.000+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:08.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:08.000+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:08.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:08.000+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:09.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:09.597+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:09.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:09.597+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:09.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:09.597+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:09.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:09.597+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:09.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:09.597+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21b8006130 2023-08-24T10:11:09.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:09.597+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b80050d0 2023-08-24T10:11:09.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:09.597+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8006130 legacy=0x7f21b800e6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:09.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:09.597+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:09.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:09.597+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:09.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:09.798+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:09.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:09.798+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8006130 legacy=0x7f21b800e6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:09.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:09.798+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:10.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:10.199+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:10.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:10.199+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8006130 legacy=0x7f21b800e6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:10.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:10.199+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:11.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:11.000+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8006130 legacy=0x7f21b800e6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:11.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:11.000+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:11.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:11.000+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:12.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:12.597+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 msgr2=0x7f21b80054e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:12.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:12.597+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80050d0 0x7f21b80054e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:12.597+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8006130 legacy=0x7f21b800e6a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:12.597+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:12.597+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21b8006130 2023-08-24T10:11:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:12.597+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b8005a20 2023-08-24T10:11:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:12.597+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:12.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:12.597+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8006130 legacy=0x7f21b80050d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:12.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:12.597+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:12.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:12.798+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:12.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:12.798+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8006130 legacy=0x7f21b80050d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:12.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:12.798+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:13.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:13.199+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:13.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:13.199+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8006130 legacy=0x7f21b80050d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:13.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:13.199+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:14.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:14.000+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:14.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:14.000+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8006130 legacy=0x7f21b80050d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:14.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:14.000+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:15.598+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:15.598+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:15.598+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8006130 legacy=0x7f21b80050d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:15.598+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:15.598+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21b800c690 2023-08-24T10:11:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:15.598+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b8005a20 2023-08-24T10:11:15.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:15.598+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:15.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:15.598+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b800c690 legacy=0x7f21b800eab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:15.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:15.598+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:15.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:15.799+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b800c690 legacy=0x7f21b800eab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:15.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:15.799+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:15.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:15.799+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:16.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:16.200+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:16.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:16.200+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b800c690 legacy=0x7f21b800eab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:16.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:16.200+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:17.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:17.001+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:17.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:17.001+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b800c690 legacy=0x7f21b800eab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:17.001+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:18.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:18.598+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:18.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:18.598+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:18.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:18.598+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b800c690 legacy=0x7f21b800eab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:18.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:18.598+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:18.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:18.598+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21b80050d0 2023-08-24T10:11:18.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:18.598+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b8005a20 2023-08-24T10:11:18.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:18.598+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:18.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:18.598+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:18.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:18.598+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:18.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:18.799+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:18.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:18.799+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:18.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:18.799+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:19.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:19.200+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:19.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:19.200+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:19.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:19.200+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:20.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:20.001+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:20.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:20.001+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:20.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:20.001+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:21.598+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:21.598+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:21.598+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:21.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:21.598+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:21.598+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21b80050d0 2023-08-24T10:11:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:21.598+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b8005a20 2023-08-24T10:11:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:21.598+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:21.598+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:21.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:21.598+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:21.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:21.799+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:21.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:21.799+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:21.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:21.799+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:22.200+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:22.200+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:22.200+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:23.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:23.001+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:23.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:23.001+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:23.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:23.001+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:24.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:24.599+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:24.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:24.599+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:24.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:24.599+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:24.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:24.599+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:24.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:24.599+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21b80050d0 2023-08-24T10:11:24.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:24.599+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b8005a20 2023-08-24T10:11:24.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:24.599+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:24.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:24.599+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:24.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:24.599+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:24.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:24.800+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:24.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:24.800+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:24.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:24.800+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:25.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:25.201+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:25.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:25.201+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:25.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:25.201+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:26.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:26.002+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:26.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:26.002+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:26.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:26.002+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:27.599+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:27.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:27.599+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:27.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:27.599+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:27.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:27.599+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:27.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:27.599+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21b80050d0 2023-08-24T10:11:27.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:27.599+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b8005a20 2023-08-24T10:11:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:27.599+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:27.599+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:27.599+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:27.800+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:27.800+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:27.800+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:28.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:28.201+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:28.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:28.201+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:28.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:28.201+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:29.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:29.002+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:29.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:29.002+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:29.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:29.002+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:30.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:30.599+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:30.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:30.599+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:30.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:30.599+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80050d0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:30.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:30.599+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:30.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:30.599+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21b80054e0 2023-08-24T10:11:30.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:30.599+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b8005a20 2023-08-24T10:11:30.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:30.599+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80054e0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:30.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:30.599+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:30.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:30.599+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:30.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:30.800+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:30.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:30.800+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80054e0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:30.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:30.800+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:31.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:31.201+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:31.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:31.201+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80054e0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:31.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:31.201+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:32.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:32.002+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:32.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:32.002+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80054e0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:32.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:32.002+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:33.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:33.599+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:33.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:33.599+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:33.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:33.599+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80054e0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:33.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:33.599+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:33.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:33.599+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21b80054e0 2023-08-24T10:11:33.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:33.599+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b8005a20 2023-08-24T10:11:33.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:33.599+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80054e0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:33.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:33.599+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:33.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:33.599+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:33.800+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:33.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:33.800+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80054e0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:33.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:33.800+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:34.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:34.201+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:34.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:34.201+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80054e0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:34.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:34.201+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:35.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:35.002+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80054e0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:35.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:35.002+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:35.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:35.002+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:36.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:36.600+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:36.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:36.600+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:36.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:36.600+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80054e0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:36.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:36.600+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:36.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:36.600+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21b80054e0 2023-08-24T10:11:36.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:36.600+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b8005a20 2023-08-24T10:11:36.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:36.600+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80054e0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:36.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:36.600+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:36.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:36.600+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:36.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:36.801+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80054e0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:36.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:36.801+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:36.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:36.801+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:37.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:37.202+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80054e0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:37.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:37.202+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:37.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:37.202+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:38.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:38.003+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80054e0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:38.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:38.003+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:38.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:38.003+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:39.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:39.600+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:39.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:39.600+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:39.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:39.600+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80054e0 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:39.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:39.600+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:39.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:39.600+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21b800a680 2023-08-24T10:11:39.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:39.600+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b8005a20 2023-08-24T10:11:39.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:39.600+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b800a680 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:39.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:39.600+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:39.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:39.600+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:39.801+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:39.801+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b800a680 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:39.801+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:40.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:40.202+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b800a680 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:40.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:40.202+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:40.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:40.202+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:41.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:41.003+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:41.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:41.003+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b800a680 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:41.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:41.003+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:42.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:42.601+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:42.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:42.601+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005a20 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:42.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:42.601+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b800a680 legacy=0x7f21b8005e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:42.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:42.601+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 0x7f21b800eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:42.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:42.601+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21b800a680 2023-08-24T10:11:42.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:42.601+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b800caa0 2023-08-24T10:11:42.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:42.601+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 msgr2=0x7f21b800eec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:42.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:42.601+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b800a680 legacy=0x7f21b8005990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:42.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:42.601+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 0x7f21b800eec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:42.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:42.802+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b800a680 legacy=0x7f21b8005990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:42.802+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 msgr2=0x7f21b800eec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:42.802+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 0x7f21b800eec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:43.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:43.203+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b800a680 legacy=0x7f21b8005990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:43.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:43.203+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 msgr2=0x7f21b800eec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:43.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:43.203+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 0x7f21b800eec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:44.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:44.003+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 msgr2=0x7f21b800eec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:44.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:44.003+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b800a680 legacy=0x7f21b8005990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:44.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:44.003+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 0x7f21b800eec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:45.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:45.600+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 msgr2=0x7f21b800eec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:45.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:45.600+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 0x7f21b800eec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:45.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:45.600+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b800a680 legacy=0x7f21b8005990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:45.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:45.600+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 0x7f21b8005990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:45.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:45.600+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21b80045b0 2023-08-24T10:11:45.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:45.600+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b800caa0 2023-08-24T10:11:45.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:45.600+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b80060a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:45.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:45.600+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 msgr2=0x7f21b8005990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:45.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:45.600+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 0x7f21b8005990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:45.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:45.801+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 msgr2=0x7f21b8005990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:45.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:45.801+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b80060a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:45.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:45.801+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 0x7f21b8005990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:46.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:46.202+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 msgr2=0x7f21b8005990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:46.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:46.202+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b80060a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:46.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:46.202+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 0x7f21b8005990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:47.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:47.003+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b80060a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:47.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:47.003+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 msgr2=0x7f21b8005990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:47.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:47.003+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 0x7f21b8005990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:48.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:48.600+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 msgr2=0x7f21b8005990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:48.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:48.600+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800caa0 0x7f21b8005990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:48.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:48.600+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b80060a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:48.600+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b800eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:48.600+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21b80045b0 2023-08-24T10:11:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:48.600+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b800a680 2023-08-24T10:11:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:48.600+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b800eec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:48.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:48.600+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:48.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:48.600+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b800eec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:48.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:48.801+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:48.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:48.801+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b800eec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:48.801+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b800eec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:49.202+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:49.202+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b800eec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:49.202+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b800eec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:50.004 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:50.003+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:50.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:50.003+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b800eec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:50.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:50.003+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b800eec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:51.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:51.601+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b800eec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:51.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:51.601+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b800eec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:51.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:51.601+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b80045b0 legacy=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:51.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:51.601+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b8005f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:51.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:51.601+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21b8005990 2023-08-24T10:11:51.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:51.601+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b800a680 2023-08-24T10:11:51.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:51.601+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8005990 legacy=0x7f21b800f400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:51.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:51.601+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:51.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:51.601+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b8005f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:51.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:51.802+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:51.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:51.802+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8005990 legacy=0x7f21b800f400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:51.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:51.802+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b8005f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:52.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:52.203+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:52.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:52.203+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8005990 legacy=0x7f21b800f400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:52.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:52.203+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b8005f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:53.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:53.004+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8005990 legacy=0x7f21b800f400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:53.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:53.004+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:53.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:53.004+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b8005f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:54.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:54.601+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:54.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:54.601+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b8005f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:54.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:54.601+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8005990 legacy=0x7f21b800f400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:54.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:54.601+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b8005f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:54.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:54.601+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21b8005990 2023-08-24T10:11:54.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:54.601+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b800a680 2023-08-24T10:11:54.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:54.601+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8005990 legacy=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:54.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:54.601+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:54.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:54.601+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b8005f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:54.802+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8005990 legacy=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:54.802+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:54.802+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b8005f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:55.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:55.203+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:55.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:55.203+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8005990 legacy=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:55.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:55.203+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b8005f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:56.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:56.004+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:56.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:56.004+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8005990 legacy=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:56.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:56.004+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b8005f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:11:57.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:57.601+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:57.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:57.601+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b8005f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:11:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:57.601+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8005990 legacy=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:11:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:57.601+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b8005f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:11:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:57.601+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21b8003cb0 2023-08-24T10:11:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:57.601+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b800a680 2023-08-24T10:11:57.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:57.601+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8003cb0 legacy=0x7f21b80040c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:57.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:57.601+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:57.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:57.601+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b8005f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:11:57.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:57.802+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:57.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:57.802+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8003cb0 legacy=0x7f21b80040c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:57.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:57.802+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b8005f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:11:58.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:58.203+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:58.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:58.203+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8003cb0 legacy=0x7f21b80040c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:58.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:58.203+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b8005f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:11:59.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:59.004+0000 7f21d659c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:11:59.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:59.004+0000 7f21d6d9d640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8003cb0 legacy=0x7f21b80040c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:11:59.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:11:59.004+0000 7f21d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b8005f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:00.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:00.602+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 msgr2=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:00.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:00.602+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b800a680 0x7f21b8005f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:12:00.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:00.602+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8003cb0 legacy=0x7f21b80040c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:00.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:00.602+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 0x7f21b80035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:00.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:00.602+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21b8003cb0 2023-08-24T10:12:00.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:00.602+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b8005990 2023-08-24T10:12:00.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:00.602+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:00.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:00.602+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8003cb0 legacy=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:00.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:00.602+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:00.803+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:00.803+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8003cb0 legacy=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:00.803+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:01.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:01.204+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:01.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:01.204+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8003cb0 legacy=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:01.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:01.204+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:02.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:02.005+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8003cb0 legacy=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:02.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:02.005+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:02.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:02.005+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:03.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:03.601+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:03.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:03.601+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:12:03.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:03.601+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8003cb0 legacy=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:03.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:03.601+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 0x7f21b80035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:03.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:03.601+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132610 con 0x7f21b8003cb0 2023-08-24T10:12:03.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:03.601+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b8005990 2023-08-24T10:12:03.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:03.602+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8003cb0 legacy=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:03.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:03.601+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:03.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:03.602+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:03.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:03.803+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8003cb0 legacy=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:03.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:03.803+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:03.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:03.803+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:04.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:04.204+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:04.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:04.204+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8003cb0 legacy=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:04.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:04.204+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:05.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:05.005+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:05.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:05.005+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8003cb0 legacy=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:05.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:05.005+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:06.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:06.603+0000 7f21d4d99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 msgr2=0x7f21b80035a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:06.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:06.603+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b8005990 0x7f21b80035a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:12:06.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:06.603+0000 7f21d4d99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b8003cb0 legacy=0x7f21b8005f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:06.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:06.603+0000 7f21d4d99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80040c0 0x7f21b80044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:06.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:06.603+0000 7f21d4d99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8132840 con 0x7f21b800f6b0 2023-08-24T10:12:06.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:06.603+0000 7f21d4d99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d809c260 con 0x7f21b80040c0 2023-08-24T10:12:06.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:06.603+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80040c0 msgr2=0x7f21b80044d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:06.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:06.603+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b800f6b0 legacy=0x7f21b8011ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:06.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:06.603+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80040c0 0x7f21b80044d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:06.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:06.804+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b800f6b0 legacy=0x7f21b8011ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:06.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:06.804+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80040c0 msgr2=0x7f21b80044d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:06.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:06.804+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80040c0 0x7f21b80044d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:07.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:07.205+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b800f6b0 legacy=0x7f21b8011ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:07.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:07.205+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80040c0 msgr2=0x7f21b80044d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:07.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:07.205+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80040c0 0x7f21b80044d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:08.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:08.006+0000 7f21d6d9d640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7f21b80040c0 msgr2=0x7f21b80044d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:08.006+0000 7f21d659c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7f21b800f6b0 legacy=0x7f21b8011ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:08.006+0000 7f21d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21b80040c0 0x7f21b80044d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:09.508 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-08-24T10:12:09.509 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-08-24T10:12:09.509 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-24T10:12:09.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:09.669+0000 7fd1a4e63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2023-08-24T10:12:09.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:09.672+0000 7fd1a4e63640 1 Processor -- start 2023-08-24T10:12:09.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:09.672+0000 7fd1a4e63640 1 -- start start 2023-08-24T10:12:09.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:09.673+0000 7fd1a4e63640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd1a0134160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:09.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:09.673+0000 7fd1a4e63640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd1a006d300 2023-08-24T10:12:09.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:09.673+0000 7fd1a4e63640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd1a0131d40 2023-08-24T10:12:09.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:09.673+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1a012bc80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:09.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:09.673+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd1a0134160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:09.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:09.673+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd1a0134160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:09.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:09.874+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1a012bc80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:09.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:09.874+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd1a0134160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:09.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:09.874+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd1a0134160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:10.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:10.275+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1a012bc80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:10.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:10.275+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd1a0134160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:10.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:10.275+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd1a0134160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:11.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:11.076+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd1a0134160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:11.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:11.076+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1a012bc80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:11.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:11.076+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd1a0134160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:12.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:12.674+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd1a0134160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:12.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:12.674+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd1a0134160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:12:12.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:12.674+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1a012bc80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:12.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:12.674+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:12.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:12.674+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd1a006d300 2023-08-24T10:12:12.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:12.674+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd1a0131d40 2023-08-24T10:12:12.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:12.674+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd180005640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:12.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:12.674+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:12.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:12.674+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:12.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:12.875+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:12.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:12.875+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd180005640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:12.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:12.875+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:13.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:13.276+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd180005640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:13.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:13.276+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:13.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:13.276+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:14.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:14.077+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd180005640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:14.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:14.077+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:14.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:14.077+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:15.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:15.674+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:15.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:15.674+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:12:15.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:15.674+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd180005640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:15.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:15.674+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:15.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:15.674+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd1a006d300 2023-08-24T10:12:15.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:15.674+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd1a0131d40 2023-08-24T10:12:15.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:15.674+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1800055e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:15.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:15.674+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:15.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:15.674+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:15.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:15.875+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:15.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:15.875+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1800055e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:15.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:15.875+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:16.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:16.276+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1800055e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:16.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:16.276+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:16.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:16.276+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:17.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:17.077+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1800055e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:17.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:17.077+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:17.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:17.077+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:18.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:18.674+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:18.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:18.674+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:12:18.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:18.674+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1800055e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:18.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:18.674+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:18.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:18.675+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd1a006d300 2023-08-24T10:12:18.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:18.675+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd1a0131d40 2023-08-24T10:12:18.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:18.675+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:18.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:18.675+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:18.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:18.675+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:18.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:18.876+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:18.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:18.876+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:18.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:18.876+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:19.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:19.277+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:19.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:19.277+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:19.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:19.277+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:20.078+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:20.078+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:20.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:20.078+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:21.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:21.675+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:21.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:21.675+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:12:21.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:21.675+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:21.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:21.675+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:21.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:21.675+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd1a006d300 2023-08-24T10:12:21.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:21.675+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd1a0131d40 2023-08-24T10:12:21.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:21.675+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:21.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:21.675+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:21.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:21.675+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:21.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:21.876+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:21.876+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:21.876+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:22.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:22.277+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:22.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:22.277+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:22.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:22.277+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:23.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:23.078+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:23.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:23.078+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:23.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:23.078+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:24.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:24.675+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:24.675+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:12:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:24.675+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1a006d300 legacy=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:24.675+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:24.675+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd1800094b0 2023-08-24T10:12:24.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:24.675+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd1a0131d40 2023-08-24T10:12:24.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:24.675+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1800094b0 legacy=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:24.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:24.675+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:24.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:24.675+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:24.876+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1800094b0 legacy=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:24.876+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:24.876+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:25.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:25.277+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:25.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:25.277+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1800094b0 legacy=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:25.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:25.277+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:26.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:26.078+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:26.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:26.078+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1800094b0 legacy=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:26.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:26.078+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:27.675+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:27.675+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:12:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:27.675+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1800094b0 legacy=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:27.675+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:27.675+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd180009a80 2023-08-24T10:12:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:27.675+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd1a0131d40 2023-08-24T10:12:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:27.675+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180009a80 legacy=0x7fd180005d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:27.675+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:27.676+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:27.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:27.877+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:27.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:27.877+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180009a80 legacy=0x7fd180005d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:27.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:27.877+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:28.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:28.278+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:28.278+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180009a80 legacy=0x7fd180005d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:28.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:28.278+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:29.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:29.079+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180009a80 legacy=0x7fd180005d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:29.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:29.079+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:29.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:29.079+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:30.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:30.676+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 msgr2=0x7fd180008df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:30.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:30.676+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1a0131d40 0x7fd180008df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:12:30.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:30.676+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180009a80 legacy=0x7fd180005d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:30.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:30.676+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 0x7fd1800055c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:30.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:30.676+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd180009a80 2023-08-24T10:12:30.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:30.676+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd180006460 2023-08-24T10:12:30.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:30.676+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180009a80 legacy=0x7fd180005d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:30.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:30.676+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 msgr2=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:30.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:30.676+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 0x7fd1800055c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:30.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:30.877+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180009a80 legacy=0x7fd180005d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:30.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:30.877+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 msgr2=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:30.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:30.877+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 0x7fd1800055c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:31.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:31.278+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 msgr2=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:31.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:31.278+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180009a80 legacy=0x7fd180005d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:31.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:31.278+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 0x7fd1800055c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:32.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:32.079+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 msgr2=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:32.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:32.079+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180009a80 legacy=0x7fd180005d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:32.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:32.079+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 0x7fd1800055c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:33.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:33.676+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 msgr2=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:33.676+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 0x7fd1800055c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:12:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:33.676+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180009a80 legacy=0x7fd180005d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:33.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:33.676+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 0x7fd1800055c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:33.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:33.676+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd180006870 2023-08-24T10:12:33.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:33.676+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd180006460 2023-08-24T10:12:33.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:33.676+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 msgr2=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:33.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:33.676+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180006870 legacy=0x7fd18000f7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:33.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:33.676+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 0x7fd1800055c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:33.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:33.877+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180006870 legacy=0x7fd18000f7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:33.877+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 msgr2=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:33.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:33.877+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 0x7fd1800055c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:34.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:34.278+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180006870 legacy=0x7fd18000f7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:34.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:34.278+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 msgr2=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:34.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:34.278+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 0x7fd1800055c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:35.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:35.079+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 msgr2=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:35.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:35.079+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180006870 legacy=0x7fd18000f7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:35.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:35.079+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 0x7fd1800055c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:36.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:36.676+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 msgr2=0x7fd1800055c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:36.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:36.676+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 0x7fd1800055c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:12:36.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:36.676+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180006870 legacy=0x7fd18000f7b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:36.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:36.676+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 0x7fd1800035b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:36.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:36.676+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd180008df0 2023-08-24T10:12:36.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:36.676+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd180006460 2023-08-24T10:12:36.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:36.676+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 msgr2=0x7fd1800035b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:36.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:36.676+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008df0 legacy=0x7fd180005d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:36.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:36.676+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 0x7fd1800035b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:36.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:36.877+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 msgr2=0x7fd1800035b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:36.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:36.877+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008df0 legacy=0x7fd180005d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:36.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:36.877+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 0x7fd1800035b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:37.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:37.278+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 msgr2=0x7fd1800035b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:37.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:37.278+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008df0 legacy=0x7fd180005d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:37.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:37.278+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 0x7fd1800035b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:38.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:38.079+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 msgr2=0x7fd1800035b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:38.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:38.079+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008df0 legacy=0x7fd180005d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:38.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:38.079+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 0x7fd1800035b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:39.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:39.677+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 msgr2=0x7fd1800035b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:39.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:39.677+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006460 0x7fd1800035b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:12:39.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:39.677+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008df0 legacy=0x7fd180005d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:39.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:39.677+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180005620 0x7fd1800035b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:39.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:39.677+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd180003af0 2023-08-24T10:12:39.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:39.677+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd180005620 2023-08-24T10:12:39.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:39.677+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003af0 legacy=0x7fd180003f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:39.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:39.677+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180005620 msgr2=0x7fd1800035b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:39.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:39.677+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180005620 0x7fd1800035b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:39.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:39.878+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003af0 legacy=0x7fd180003f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:39.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:39.878+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180005620 msgr2=0x7fd1800035b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:39.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:39.878+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180005620 0x7fd1800035b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:40.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:40.279+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003af0 legacy=0x7fd180003f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:40.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:40.279+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180005620 msgr2=0x7fd1800035b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:40.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:40.279+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180005620 0x7fd1800035b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:41.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:41.080+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180005620 msgr2=0x7fd1800035b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:41.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:41.080+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003af0 legacy=0x7fd180003f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:41.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:41.080+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180005620 0x7fd1800035b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:42.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:42.677+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180005620 msgr2=0x7fd1800035b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:42.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:42.677+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180005620 0x7fd1800035b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:12:42.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:42.677+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003af0 legacy=0x7fd180003f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:42.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:42.677+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 0x7fd18000a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:42.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:42.677+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd180003af0 2023-08-24T10:12:42.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:42.677+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd180005cd0 2023-08-24T10:12:42.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:42.677+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003af0 legacy=0x7fd18000aac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:42.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:42.677+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 msgr2=0x7fd18000a580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:42.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:42.677+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 0x7fd18000a580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:42.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:42.878+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003af0 legacy=0x7fd18000aac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:42.878+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 msgr2=0x7fd18000a580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:42.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:42.878+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 0x7fd18000a580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:43.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:43.279+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003af0 legacy=0x7fd18000aac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:43.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:43.279+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 msgr2=0x7fd18000a580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:43.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:43.279+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 0x7fd18000a580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:44.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:44.080+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 msgr2=0x7fd18000a580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:44.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:44.080+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003af0 legacy=0x7fd18000aac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:44.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:44.080+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 0x7fd18000a580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:45.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:45.677+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 msgr2=0x7fd18000a580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:45.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:45.677+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 0x7fd18000a580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:12:45.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:45.677+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003af0 legacy=0x7fd18000aac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:45.677+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 0x7fd18000a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:45.677+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd180003590 2023-08-24T10:12:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:45.677+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd180005cd0 2023-08-24T10:12:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:45.677+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003590 legacy=0x7fd180004f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:45.677+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 msgr2=0x7fd18000a580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:45.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:45.677+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 0x7fd18000a580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:45.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:45.878+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003590 legacy=0x7fd180004f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:45.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:45.878+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 msgr2=0x7fd18000a580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:45.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:45.878+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 0x7fd18000a580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:46.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:46.279+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003590 legacy=0x7fd180004f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:46.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:46.279+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 msgr2=0x7fd18000a580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:46.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:46.279+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 0x7fd18000a580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:47.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:47.080+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 msgr2=0x7fd18000a580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:47.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:47.080+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003590 legacy=0x7fd180004f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:47.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:47.080+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 0x7fd18000a580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:48.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:48.677+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 msgr2=0x7fd18000a580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:48.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:48.677+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180005cd0 0x7fd18000a580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:12:48.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:48.677+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003590 legacy=0x7fd180004f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:48.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:48.677+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 0x7fd18000aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:48.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:48.677+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd180003590 2023-08-24T10:12:48.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:48.677+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd18000aac0 2023-08-24T10:12:48.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:48.677+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 msgr2=0x7fd18000aed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:48.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:48.677+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003590 legacy=0x7fd180003f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:48.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:48.677+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 0x7fd18000aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:48.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:48.878+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003590 legacy=0x7fd180003f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:48.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:48.879+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 msgr2=0x7fd18000aed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:48.879+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 0x7fd18000aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:49.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:49.279+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003590 legacy=0x7fd180003f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:49.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:49.280+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 msgr2=0x7fd18000aed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:49.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:49.280+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 0x7fd18000aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:50.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:50.080+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003590 legacy=0x7fd180003f00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:50.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:50.081+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 msgr2=0x7fd18000aed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:50.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:50.081+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 0x7fd18000aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:51.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:51.678+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 msgr2=0x7fd18000aed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:51.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:51.678+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 0x7fd18000aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:12:51.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:51.678+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180003590 legacy=0x7fd180003f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:51.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:51.678+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 0x7fd18000aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:51.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:51.678+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd18000a580 2023-08-24T10:12:51.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:51.678+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd18000aac0 2023-08-24T10:12:51.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:51.678+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000a580 legacy=0x7fd1800117c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:51.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:51.678+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 msgr2=0x7fd18000aed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:51.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:51.678+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 0x7fd18000aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:51.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:51.879+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000a580 legacy=0x7fd1800117c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:51.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:51.879+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 msgr2=0x7fd18000aed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:51.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:51.879+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 0x7fd18000aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:52.280+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 msgr2=0x7fd18000aed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:52.280+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000a580 legacy=0x7fd1800117c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:52.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:52.280+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 0x7fd18000aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:53.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:53.081+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000a580 legacy=0x7fd1800117c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:53.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:53.081+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 msgr2=0x7fd18000aed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:53.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:53.081+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 0x7fd18000aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:54.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:54.678+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 msgr2=0x7fd18000aed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:54.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:54.678+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aac0 0x7fd18000aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:12:54.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:54.678+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000a580 legacy=0x7fd1800117c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:54.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:54.678+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180003f00 0x7fd180004310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:54.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:54.678+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd180004850 2023-08-24T10:12:54.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:54.678+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd180003f00 2023-08-24T10:12:54.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:54.678+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004850 legacy=0x7fd180004c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:54.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:54.678+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180003f00 msgr2=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:54.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:54.678+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180003f00 0x7fd180004310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:54.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:54.879+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180003f00 msgr2=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:54.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:54.879+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004850 legacy=0x7fd180004c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:54.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:54.879+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180003f00 0x7fd180004310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:55.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:55.280+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180003f00 msgr2=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:55.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:55.280+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004850 legacy=0x7fd180004c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:55.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:55.280+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180003f00 0x7fd180004310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:56.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:56.081+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180003f00 msgr2=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:56.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:56.081+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004850 legacy=0x7fd180004c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:56.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:56.081+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180003f00 0x7fd180004310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:12:57.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:57.678+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180003f00 msgr2=0x7fd180004310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:57.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:57.678+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180003f00 0x7fd180004310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:12:57.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:57.678+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004850 legacy=0x7fd180004c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:12:57.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:57.678+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aa30 0x7fd18000ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:12:57.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:57.678+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd180007dd0 2023-08-24T10:12:57.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:57.678+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd18000aa30 2023-08-24T10:12:57.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:57.678+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180007dd0 legacy=0x7fd1800081e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:57.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:57.678+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aa30 msgr2=0x7fd18000ae40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:57.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:57.678+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aa30 0x7fd18000ae40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:12:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:57.879+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aa30 msgr2=0x7fd18000ae40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:57.879+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180007dd0 legacy=0x7fd1800081e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:57.879+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aa30 0x7fd18000ae40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:12:58.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:58.280+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180007dd0 legacy=0x7fd1800081e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:58.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:58.280+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aa30 msgr2=0x7fd18000ae40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:58.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:58.280+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aa30 0x7fd18000ae40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:12:59.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:59.081+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180007dd0 legacy=0x7fd1800081e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:12:59.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:59.081+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aa30 msgr2=0x7fd18000ae40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:12:59.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:12:59.081+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aa30 0x7fd18000ae40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:00.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:00.679+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aa30 msgr2=0x7fd18000ae40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:00.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:00.679+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000aa30 0x7fd18000ae40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:00.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:00.679+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180007dd0 legacy=0x7fd1800081e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:00.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:00.679+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 0x7fd180006dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:00.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:00.679+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd18000ae40 2023-08-24T10:13:00.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:00.679+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd1800088f0 2023-08-24T10:13:00.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:00.679+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 msgr2=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:00.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:00.679+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180007300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:00.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:00.679+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 0x7fd180006dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:00.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:00.880+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 msgr2=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:00.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:00.880+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180007300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:00.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:00.880+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 0x7fd180006dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:01.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:01.281+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 msgr2=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:01.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:01.281+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180007300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:01.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:01.281+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 0x7fd180006dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:02.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:02.082+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 msgr2=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:02.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:02.082+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180007300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:02.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:02.082+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 0x7fd180006dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:03.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:03.679+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 msgr2=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:03.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:03.679+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 0x7fd180006dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:03.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:03.679+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180007300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:03.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:03.679+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 0x7fd180006dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:03.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:03.679+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd180007a10 2023-08-24T10:13:03.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:03.679+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd1800088f0 2023-08-24T10:13:03.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:03.679+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 msgr2=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:03.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:03.679+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180007a10 legacy=0x7fd180007e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:03.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:03.679+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 0x7fd180006dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:03.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:03.880+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 msgr2=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:03.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:03.880+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180007a10 legacy=0x7fd180007e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:03.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:03.880+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 0x7fd180006dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:04.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:04.281+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 msgr2=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:04.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:04.281+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180007a10 legacy=0x7fd180007e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:04.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:04.281+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 0x7fd180006dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:05.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:05.082+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 msgr2=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:05.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:05.082+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180007a10 legacy=0x7fd180007e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:05.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:05.082+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 0x7fd180006dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:06.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:06.679+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 msgr2=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:06.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:06.679+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 0x7fd180006dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:06.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:06.679+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180007a10 legacy=0x7fd180007e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:06.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:06.679+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 0x7fd180006dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:06.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:06.679+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd18000ae40 2023-08-24T10:13:06.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:06.679+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd1800088f0 2023-08-24T10:13:06.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:06.679+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 msgr2=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:06.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:06.679+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180007300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:06.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:06.679+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 0x7fd180006dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:06.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:06.880+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180007300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:06.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:06.880+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 msgr2=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:06.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:06.880+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 0x7fd180006dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:07.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:07.281+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 msgr2=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:07.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:07.281+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180007300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:07.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:07.281+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 0x7fd180006dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:08.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:08.082+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180007300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:08.082+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 msgr2=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:08.082+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 0x7fd180006dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:09.679+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 msgr2=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:09.679+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800088f0 0x7fd180006dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:09.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:09.679+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180007300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:09.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:09.679+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 0x7fd180007e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:09.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:09.679+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd18000ae40 2023-08-24T10:13:09.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:09.679+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd180007a10 2023-08-24T10:13:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:09.679+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:09.679+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 msgr2=0x7fd180007e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:09.679+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 0x7fd180007e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:09.880+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 msgr2=0x7fd180007e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:09.880+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:09.880+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 0x7fd180007e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:10.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:10.281+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 msgr2=0x7fd180007e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:10.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:10.281+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:10.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:10.281+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 0x7fd180007e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:11.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:11.082+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 msgr2=0x7fd180007e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:11.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:11.082+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:11.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:11.082+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 0x7fd180007e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:12.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:12.680+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 msgr2=0x7fd180007e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:12.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:12.680+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 0x7fd180007e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:12.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:12.680+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:12.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:12.680+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 0x7fd180007e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:12.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:12.680+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd180006dc0 2023-08-24T10:13:12.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:12.680+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd180007a10 2023-08-24T10:13:12.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:12.680+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180006dc0 legacy=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:12.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:12.680+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 msgr2=0x7fd180007e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:12.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:12.680+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 0x7fd180007e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:12.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:12.881+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180006dc0 legacy=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:12.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:12.881+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 msgr2=0x7fd180007e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:12.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:12.881+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 0x7fd180007e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:13.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:13.282+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180006dc0 legacy=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:13.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:13.282+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 msgr2=0x7fd180007e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:13.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:13.282+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 0x7fd180007e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:14.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:14.083+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 msgr2=0x7fd180007e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:14.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:14.083+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180006dc0 legacy=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:14.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:14.083+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 0x7fd180007e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:15.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:15.680+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 msgr2=0x7fd180007e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:15.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:15.680+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180007a10 0x7fd180007e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:15.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:15.680+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180006dc0 legacy=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:15.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:15.680+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 0x7fd180016c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:15.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:15.680+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd18000ae40 2023-08-24T10:13:15.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:15.680+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd1800147e0 2023-08-24T10:13:15.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:15.680+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:15.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:15.680+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 msgr2=0x7fd180016c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:15.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:15.680+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 0x7fd180016c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:15.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:15.881+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:15.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:15.881+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 msgr2=0x7fd180016c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:15.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:15.881+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 0x7fd180016c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:16.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:16.282+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:16.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:16.282+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 msgr2=0x7fd180016c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:16.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:16.282+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 0x7fd180016c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:17.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:17.083+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 msgr2=0x7fd180016c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:17.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:17.083+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:17.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:17.083+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 0x7fd180016c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:18.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:18.680+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 msgr2=0x7fd180016c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:18.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:18.680+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 0x7fd180016c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:18.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:18.680+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:18.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:18.680+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:18.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:18.680+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd18000ae40 2023-08-24T10:13:18.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:18.680+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd180006dc0 2023-08-24T10:13:18.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:18.680+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:18.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:18.680+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:18.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:18.680+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:18.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:18.881+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:18.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:18.881+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:18.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:18.881+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:19.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:19.282+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:19.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:19.282+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:19.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:19.282+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:20.083+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:20.083+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:20.083+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:21.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:21.681+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:21.681+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:21.681+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180004310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:21.681+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 0x7fd180004a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:21.681+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd180004f60 2023-08-24T10:13:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:21.681+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd180008720 2023-08-24T10:13:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:21.681+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 msgr2=0x7fd180004a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:21.681+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 0x7fd180004a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:21.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:21.681+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:21.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:21.882+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 msgr2=0x7fd180004a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:21.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:21.882+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:21.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:21.882+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 0x7fd180004a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:22.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:22.283+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 msgr2=0x7fd180004a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:22.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:22.283+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:22.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:22.283+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 0x7fd180004a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:23.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:23.084+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:23.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:23.084+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 msgr2=0x7fd180004a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:23.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:23.084+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 0x7fd180004a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:24.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:24.681+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 msgr2=0x7fd180004a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:24.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:24.681+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 0x7fd180004a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:24.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:24.681+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:24.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:24.681+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 0x7fd180004a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:24.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:24.681+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd18000ae40 2023-08-24T10:13:24.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:24.681+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd180008720 2023-08-24T10:13:24.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:24.681+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:24.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:24.681+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 msgr2=0x7fd180004a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:24.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:24.681+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 0x7fd180004a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:24.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:24.882+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:24.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:24.882+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 msgr2=0x7fd180004a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:24.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:24.882+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 0x7fd180004a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:25.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:25.283+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:25.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:25.283+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 msgr2=0x7fd180004a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:25.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:25.283+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 0x7fd180004a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:26.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:26.084+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:26.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:26.084+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 msgr2=0x7fd180004a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:26.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:26.084+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 0x7fd180004a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:27.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:27.681+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 msgr2=0x7fd180004a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:27.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:27.681+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180008720 0x7fd180004a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:27.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:27.681+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180004310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:27.681+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 0x7fd180016c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:27.681+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd180004f60 2023-08-24T10:13:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:27.681+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd1800147e0 2023-08-24T10:13:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:27.681+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:27.681+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 msgr2=0x7fd180016c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:27.681+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 0x7fd180016c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:27.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:27.882+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:27.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:27.882+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 msgr2=0x7fd180016c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:27.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:27.882+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 0x7fd180016c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:28.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:28.283+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 msgr2=0x7fd180016c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:28.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:28.283+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:28.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:28.283+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 0x7fd180016c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:29.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:29.084+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:29.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:29.084+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 msgr2=0x7fd180016c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:29.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:29.084+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 0x7fd180016c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:30.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:30.681+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 msgr2=0x7fd180016c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:30.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:30.681+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 0x7fd180016c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:30.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:30.681+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:30.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:30.681+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800137d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:30.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:30.681+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd18000ae40 2023-08-24T10:13:30.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:30.681+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd180006dc0 2023-08-24T10:13:30.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:30.681+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:30.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:30.681+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800137d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:30.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:30.681+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800137d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:30.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:30.882+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:30.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:30.882+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800137d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:30.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:30.882+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800137d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:31.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:31.283+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:31.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:31.283+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800137d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:31.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:31.283+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800137d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:32.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:32.084+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:32.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:32.084+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800137d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:32.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:32.084+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800137d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:33.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:33.682+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800137d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:33.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:33.682+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800137d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:33.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:33.682+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:33.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:33.682+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000f7b0 0x7fd180011bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:33.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:33.682+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd180013d10 2023-08-24T10:13:33.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:33.682+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd18000f7b0 2023-08-24T10:13:33.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:33.682+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180013d10 legacy=0x7fd180014120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:33.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:33.682+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000f7b0 msgr2=0x7fd180011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:33.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:33.682+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000f7b0 0x7fd180011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:33.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:33.883+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000f7b0 msgr2=0x7fd180011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:33.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:33.883+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180013d10 legacy=0x7fd180014120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:33.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:33.883+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000f7b0 0x7fd180011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:34.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:34.284+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000f7b0 msgr2=0x7fd180011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:34.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:34.284+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180013d10 legacy=0x7fd180014120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:34.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:34.284+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000f7b0 0x7fd180011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:35.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:35.085+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180013d10 legacy=0x7fd180014120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:35.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:35.085+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000f7b0 msgr2=0x7fd180011bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:35.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:35.085+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000f7b0 0x7fd180011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:36.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:36.682+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000f7b0 msgr2=0x7fd180011bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:36.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:36.682+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000f7b0 0x7fd180011bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:36.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:36.682+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180013d10 legacy=0x7fd180014120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:36.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:36.682+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180008630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:36.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:36.682+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd180013d10 2023-08-24T10:13:36.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:36.682+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd18000ae40 2023-08-24T10:13:36.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:36.682+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180013d10 legacy=0x7fd180014120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:36.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:36.682+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 msgr2=0x7fd180008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:36.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:36.682+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:36.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:36.883+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 msgr2=0x7fd180008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:36.883+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180013d10 legacy=0x7fd180014120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:36.883+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:37.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:37.284+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 msgr2=0x7fd180008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:37.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:37.284+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180013d10 legacy=0x7fd180014120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:37.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:37.284+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:38.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:38.085+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 msgr2=0x7fd180008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:38.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:38.085+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180013d10 legacy=0x7fd180014120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:38.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:38.085+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:39.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:39.682+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 msgr2=0x7fd180008630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:39.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:39.682+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:39.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:39.682+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180013d10 legacy=0x7fd180014120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:39.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:39.682+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 0x7fd180007cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:39.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:39.682+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd180006dc0 2023-08-24T10:13:39.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:39.682+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd1800078e0 2023-08-24T10:13:39.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:39.682+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180006dc0 legacy=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:39.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:39.682+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 msgr2=0x7fd180007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:39.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:39.682+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 0x7fd180007cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:39.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:39.883+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180006dc0 legacy=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:39.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:39.883+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 msgr2=0x7fd180007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:39.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:39.883+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 0x7fd180007cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:40.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:40.283+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 msgr2=0x7fd180007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:40.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:40.283+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180006dc0 legacy=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:40.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:40.283+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 0x7fd180007cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:41.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:41.084+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 msgr2=0x7fd180007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:41.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:41.084+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180006dc0 legacy=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:41.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:41.084+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 0x7fd180007cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:42.682+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 msgr2=0x7fd180007cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:42.682+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 0x7fd180007cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:42.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:42.682+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180006dc0 legacy=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:42.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:42.682+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 0x7fd180007cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:42.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:42.682+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd18000ae40 2023-08-24T10:13:42.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:42.682+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd1800078e0 2023-08-24T10:13:42.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:42.682+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:42.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:42.682+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 msgr2=0x7fd180007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:42.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:42.682+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 0x7fd180007cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:42.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:42.883+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:42.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:42.883+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 msgr2=0x7fd180007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:42.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:42.883+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 0x7fd180007cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:43.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:43.284+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:43.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:43.284+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 msgr2=0x7fd180007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:43.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:43.284+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 0x7fd180007cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:44.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:44.085+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 msgr2=0x7fd180007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:44.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:44.085+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:44.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:44.085+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 0x7fd180007cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:45.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:45.682+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 msgr2=0x7fd180007cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:45.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:45.682+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800078e0 0x7fd180007cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:45.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:45.682+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180008600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:45.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:45.682+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:45.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:45.682+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd1800137d0 2023-08-24T10:13:45.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:45.682+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd180006dc0 2023-08-24T10:13:45.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:45.682+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:45.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:45.682+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1800137d0 legacy=0x7fd180015bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:45.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:45.682+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:45.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:45.883+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:45.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:45.883+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1800137d0 legacy=0x7fd180015bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:45.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:45.883+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:46.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:46.284+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:46.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:46.284+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1800137d0 legacy=0x7fd180015bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:46.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:46.284+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:47.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:47.085+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:47.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:47.085+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1800137d0 legacy=0x7fd180015bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:47.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:47.085+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:48.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:48.682+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:48.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:48.682+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:48.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:48.682+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd1800137d0 legacy=0x7fd180015bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:48.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:48.682+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:48.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:48.682+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd18000ae40 2023-08-24T10:13:48.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:48.682+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd180006dc0 2023-08-24T10:13:48.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:48.682+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:48.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:48.682+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:48.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:48.682+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:48.883+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:48.883+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:48.883+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:49.284+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:49.284+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:49.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:49.284+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:50.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:50.085+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:50.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:50.085+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:50.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:50.085+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:51.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:51.682+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:51.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:51.682+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:51.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:51.682+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd18000ae40 legacy=0x7fd180004310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:51.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:51.682+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:51.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:51.682+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd180008780 2023-08-24T10:13:51.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:51.682+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd180006dc0 2023-08-24T10:13:51.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:51.682+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008780 legacy=0x7fd180004a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:51.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:51.682+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:51.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:51.682+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:51.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:51.883+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008780 legacy=0x7fd180004a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:51.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:51.883+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:51.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:51.883+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:52.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:52.284+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008780 legacy=0x7fd180004a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:52.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:52.284+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:52.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:52.284+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:53.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:53.085+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:53.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:53.085+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008780 legacy=0x7fd180004a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:53.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:53.085+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:54.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:54.683+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 msgr2=0x7fd1800071d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:54.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:54.683+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180006dc0 0x7fd1800071d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:54.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:54.683+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008780 legacy=0x7fd180004a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:54.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:54.683+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180004310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:54.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:54.683+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd180008780 2023-08-24T10:13:54.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:54.683+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd18000ae40 2023-08-24T10:13:54.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:54.683+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008780 legacy=0x7fd180004850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:54.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:54.683+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 msgr2=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:54.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:54.683+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180004310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:54.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:54.884+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008780 legacy=0x7fd180004850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:54.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:54.884+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 msgr2=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:54.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:54.884+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180004310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:55.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:55.285+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 msgr2=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:55.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:55.285+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008780 legacy=0x7fd180004850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:55.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:55.285+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180004310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:56.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:56.086+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 msgr2=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:56.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:56.086+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008780 legacy=0x7fd180004850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:56.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:56.086+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180004310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:13:57.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:57.683+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 msgr2=0x7fd180004310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:57.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:57.683+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180004310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:13:57.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:57.683+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008780 legacy=0x7fd180004850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:13:57.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:57.683+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 0x7fd180016c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:13:57.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:57.683+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd180004f60 2023-08-24T10:13:57.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:57.683+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd1800147e0 2023-08-24T10:13:57.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:57.683+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:57.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:57.683+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 msgr2=0x7fd180016c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:57.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:57.683+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 0x7fd180016c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:13:57.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:57.884+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:57.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:57.884+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 msgr2=0x7fd180016c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:57.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:57.884+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 0x7fd180016c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:13:58.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:58.285+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:58.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:58.285+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 msgr2=0x7fd180016c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:58.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:58.285+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 0x7fd180016c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:13:59.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:59.086+0000 7fd197fff640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 msgr2=0x7fd180016c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:13:59.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:59.086+0000 7fd19e59c640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:13:59.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:13:59.086+0000 7fd197fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 0x7fd180016c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:14:00.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:00.682+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 msgr2=0x7fd180016c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:14:00.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:00.682+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd1800147e0 0x7fd180016c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:14:00.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:00.682+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180006dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:14:00.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:00.682+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180008780 0x7fd180004310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:14:00.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:00.682+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd180004f60 2023-08-24T10:14:00.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:00.682+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd180008780 2023-08-24T10:14:00.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:00.683+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180004850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:14:00.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:00.683+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180008780 msgr2=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:14:00.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:00.683+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180008780 0x7fd180004310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:14:00.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:00.884+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180008780 msgr2=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:14:00.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:00.884+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180004850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:14:00.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:00.884+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180008780 0x7fd180004310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:14:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:01.285+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180004850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:14:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:01.285+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180008780 msgr2=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:14:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:01.285+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180008780 0x7fd180004310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:14:02.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:02.086+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180004850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:14:02.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:02.086+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180008780 msgr2=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:14:02.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:02.086+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180008780 0x7fd180004310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:14:03.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:03.684+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd180008780 msgr2=0x7fd180004310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:14:03.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:03.684+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd180008780 0x7fd180004310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:14:03.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:03.684+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180004f60 legacy=0x7fd180004850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:14:03.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:03.684+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180007dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:14:03.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:03.684+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0099440 con 0x7fd180008310 2023-08-24T10:14:03.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:03.684+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd18000ae40 2023-08-24T10:14:03.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:03.684+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 msgr2=0x7fd180007dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:14:03.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:03.684+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008310 legacy=0x7fd1800157e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:14:03.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:03.684+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180007dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:14:03.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:03.885+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 msgr2=0x7fd180007dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:14:03.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:03.885+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008310 legacy=0x7fd1800157e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:14:03.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:03.885+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180007dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:14:04.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:04.286+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008310 legacy=0x7fd1800157e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:14:04.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:04.286+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 msgr2=0x7fd180007dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:14:04.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:04.286+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180007dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:14:05.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:05.087+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 msgr2=0x7fd180007dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:14:05.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:05.087+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008310 legacy=0x7fd1800157e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:14:05.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:05.087+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180007dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:14:06.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:06.684+0000 7fd19cd99640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 msgr2=0x7fd180007dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:14:06.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:06.684+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180007dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-24T10:14:06.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:06.684+0000 7fd19cd99640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008310 legacy=0x7fd1800157e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-24T10:14:06.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:06.684+0000 7fd19cd99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180004310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-24T10:14:06.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:06.684+0000 7fd19cd99640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a012b9d0 con 0x7fd180008780 2023-08-24T10:14:06.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:06.684+0000 7fd19cd99640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a009a050 con 0x7fd18000ae40 2023-08-24T10:14:06.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:06.684+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 msgr2=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:14:06.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:06.684+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008780 legacy=0x7fd180004850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:14:06.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:06.684+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180004310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-24T10:14:06.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:06.885+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008780 legacy=0x7fd180004850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:14:06.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:06.885+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 msgr2=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:14:06.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:06.885+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180004310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-24T10:14:07.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:07.286+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 msgr2=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:14:07.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:07.286+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008780 legacy=0x7fd180004850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:14:07.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:07.286+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180004310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-08-24T10:14:08.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:08.087+0000 7fd19e59c640 1 -- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 msgr2=0x7fd180004310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.60:3300/0 2023-08-24T10:14:08.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:08.087+0000 7fd197fff640 1 -- >> v1:172.21.15.60:6789/0 conn(0x7fd180008780 legacy=0x7fd180004850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.60:6789/0 2023-08-24T10:14:08.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-08-24T10:14:08.087+0000 7fd19e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd18000ae40 0x7fd180004310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-08-24T10:14:09.591 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-08-24T10:14:09.592 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1938, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1496, in healthy manager.wait_for_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2743, in wait_for_clean while not self.is_clean(): File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2679, in is_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-08-24T10:14:09.593 INFO:teuthology.misc:Shutting down mds daemons... 2023-08-24T10:14:09.595 INFO:teuthology.misc:Shutting down osd daemons... 2023-08-24T10:14:09.596 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-08-24T10:14:09.597 INFO:teuthology.orchestra.run:waiting for 300 2023-08-24T10:14:09.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-24T10:14:09.598 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1938, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1496, in healthy manager.wait_for_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2743, in wait_for_clean while not self.is_clean(): File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2679, in is_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log' 2023-08-24T10:14:09.598 INFO:tasks.ceph.osd.0:Stopped 2023-08-24T10:14:09.599 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-08-24T10:14:09.599 INFO:teuthology.orchestra.run:waiting for 300 2023-08-24T10:14:09.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-24T10:14:09.600 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1938, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1496, in healthy manager.wait_for_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2743, in wait_for_clean while not self.is_clean(): File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2679, in is_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log' 2023-08-24T10:14:09.601 INFO:tasks.ceph.osd.1:Stopped 2023-08-24T10:14:09.601 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-08-24T10:14:09.602 INFO:teuthology.orchestra.run:waiting for 300 2023-08-24T10:14:09.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-24T10:14:09.602 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1938, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1496, in healthy manager.wait_for_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2743, in wait_for_clean while not self.is_clean(): File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2679, in is_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi135 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log' 2023-08-24T10:14:09.604 INFO:tasks.ceph.osd.2:Stopped 2023-08-24T10:14:09.605 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-08-24T10:14:09.606 INFO:teuthology.orchestra.run:waiting for 300 2023-08-24T10:14:09.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-24T10:14:09.608 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1938, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1496, in healthy manager.wait_for_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2743, in wait_for_clean while not self.is_clean(): File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2679, in is_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi135 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log' 2023-08-24T10:14:09.609 INFO:tasks.ceph.osd.3:Stopped 2023-08-24T10:14:09.609 INFO:teuthology.misc:Shutting down mgr daemons... 2023-08-24T10:14:09.610 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-08-24T10:14:09.610 INFO:teuthology.orchestra.run:waiting for 300 2023-08-24T10:14:09.611 INFO:tasks.ceph.mgr.x:Stopped 2023-08-24T10:14:09.611 INFO:teuthology.misc:Shutting down mon daemons... 2023-08-24T10:14:09.612 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-08-24T10:14:09.612 INFO:teuthology.orchestra.run:waiting for 300 2023-08-24T10:14:09.613 INFO:tasks.ceph.mon.a:Stopped 2023-08-24T10:14:09.613 INFO:tasks.ceph:Checking cluster log for badness... 2023-08-24T10:14:09.614 DEBUG:teuthology.orchestra.run.smithi060:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2023-08-24T10:14:09.653 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi060.front.sepia.ceph.com 2023-08-24T10:14:09.654 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-08-24T10:14:09.919 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi060.front.sepia.ceph.com 2023-08-24T10:14:09.920 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-08-24T10:14:10.001 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi135.front.sepia.ceph.com 2023-08-24T10:14:10.002 DEBUG:teuthology.orchestra.run.smithi135:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-08-24T10:14:10.115 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi135.front.sepia.ceph.com 2023-08-24T10:14:10.115 DEBUG:teuthology.orchestra.run.smithi135:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-08-24T10:14:10.235 INFO:tasks.ceph:Archiving mon data... 2023-08-24T10:14:10.236 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi/7378498/data/mon.a.tgz 2023-08-24T10:14:10.237 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2023-08-24T10:14:10.256 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.2S95nS8vel 2023-08-24T10:14:10.256 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.2S95nS8vel 2023-08-24T10:14:10.456 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.2S95nS8vel 2023-08-24T10:14:10.548 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.2S95nS8vel is 190KB 2023-08-24T10:14:10.600 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.2S95nS8vel 2023-08-24T10:14:10.615 INFO:tasks.ceph:Cleaning ceph cluster... 2023-08-24T10:14:10.616 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-08-24T10:14:10.657 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-08-24T10:14:10.688 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-08-24T10:14:10.689 DEBUG:teuthology.orchestra.run.smithi060:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-08-24T10:14:10.728 DEBUG:teuthology.orchestra.run.smithi135:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-08-24T10:14:10.758 INFO:teuthology.orchestra.run.smithi060.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-08-24T10:14:10.762 INFO:teuthology.orchestra.run.smithi135.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-08-24T10:14:10.767 INFO:tasks.ceph:Archiving crash dumps... 2023-08-24T10:14:10.769 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/crash to /home/teuthworker/archive/matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi/7378498/remote/smithi060/crash 2023-08-24T10:14:10.770 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-08-24T10:14:10.834 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/lib/ceph/crash to /home/teuthworker/archive/matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi/7378498/remote/smithi135/crash 2023-08-24T10:14:10.835 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-08-24T10:14:10.889 INFO:tasks.ceph:Compressing logs... 2023-08-24T10:14:10.889 DEBUG:teuthology.orchestra.run.smithi060:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-24T10:14:10.892 DEBUG:teuthology.orchestra.run.smithi135:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-24T10:14:10.921 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84583.log 2023-08-24T10:14:10.922 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91307.log 2023-08-24T10:14:10.922 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89039.log 2023-08-24T10:14:10.922 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84583.log: gzip/var/log/ceph/ceph-client.admin.91307.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.90047.log 2023-08-24T10:14:10.922 INFO:teuthology.orchestra.run.smithi060.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.84583.log.gz 2023-08-24T10:14:10.922 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89039.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.91307.log.gz 2023-08-24T10:14:10.922 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89585.log 2023-08-24T10:14:10.922 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.89039.log.gz 2023-08-24T10:14:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88574.log 2023-08-24T10:14:10.926 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-08-24T10:14:10.926 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82770.log 2023-08-24T10:14:10.926 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82679.log 2023-08-24T10:14:10.926 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.82770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82721.log 2023-08-24T10:14:10.926 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.82679.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.82770.log.gz 2023-08-24T10:14:10.927 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-08-24T10:14:10.927 INFO:teuthology.orchestra.run.smithi135.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.82679.log.gz 2023-08-24T10:14:10.927 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82810.log 2023-08-24T10:14:10.927 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.82721.log: /var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-08-24T10:14:10.927 INFO:teuthology.orchestra.run.smithi135.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.82721.log.gz 2023-08-24T10:14:10.927 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/ceph-client.admin.82810.log: /var/log/ceph/ceph-osd.3.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.82810.log.gz 2023-08-24T10:14:10.929 INFO:teuthology.orchestra.run.smithi135.stderr: 90.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-08-24T10:14:10.941 INFO:teuthology.orchestra.run.smithi135.stderr: 94.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-08-24T10:14:10.960 INFO:teuthology.orchestra.run.smithi135.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-08-24T10:14:10.960 INFO:teuthology.orchestra.run.smithi135.stderr: 2023-08-24T10:14:10.960 INFO:teuthology.orchestra.run.smithi135.stderr:real 0m0.051s 2023-08-24T10:14:10.961 INFO:teuthology.orchestra.run.smithi135.stderr:user 0m0.070s 2023-08-24T10:14:10.961 INFO:teuthology.orchestra.run.smithi135.stderr:sys 0m0.020s 2023-08-24T10:14:10.976 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.90047.log.gz 2023-08-24T10:14:10.977 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90509.log 2023-08-24T10:14:10.977 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90299.log 2023-08-24T10:14:10.977 INFO:teuthology.orchestra.run.smithi060.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.89585.log.gz 2023-08-24T10:14:10.977 INFO:teuthology.orchestra.run.smithi060.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.88574.log.gz 2023-08-24T10:14:10.977 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90257.log 2023-08-24T10:14:10.977 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90299.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.90509.log.gz 2023-08-24T10:14:10.977 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85087.log 2023-08-24T10:14:10.978 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0%/var/log/ceph/ceph-client.admin.90257.log: -- replaced with /var/log/ceph/ceph-client.admin.90299.log.gz 2023-08-24T10:14:10.978 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88364.log 2023-08-24T10:14:10.978 INFO:teuthology.orchestra.run.smithi060.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.90257.log.gz 2023-08-24T10:14:10.978 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87944.log 2023-08-24T10:14:10.978 INFO:teuthology.orchestra.run.smithi060.stderr:gzip 86.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.85087.log.gz -- 2023-08-24T10:14:10.978 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.88280.log 2023-08-24T10:14:10.978 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88364.log: /var/log/ceph/ceph-client.admin.87944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90845.log 2023-08-24T10:14:10.978 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.88364.log.gz 2023-08-24T10:14:10.979 INFO:teuthology.orchestra.run.smithi060.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.87944.log.gz 2023-08-24T10:14:10.979 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91645.log 2023-08-24T10:14:10.979 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90845.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.88280.log.gz 2023-08-24T10:14:10.979 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84481.log 2023-08-24T10:14:10.979 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.90845.log.gz 2023-08-24T10:14:10.979 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91265.log 2023-08-24T10:14:10.979 INFO:teuthology.orchestra.run.smithi060.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.91645.log.gz 2023-08-24T10:14:10.980 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84481.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-08-24T10:14:10.980 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83072.log 2023-08-24T10:14:10.980 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91265.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.84481.log.gz 2023-08-24T10:14:10.980 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87776.log 2023-08-24T10:14:10.980 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.0.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.91265.log.gz 2023-08-24T10:14:10.980 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87734.log 2023-08-24T10:14:10.980 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83072.log: /var/log/ceph/ceph-client.admin.87776.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.83072.log.gz 2023-08-24T10:14:10.980 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-08-24T10:14:10.981 INFO:teuthology.orchestra.run.smithi060.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.87776.log.gz 2023-08-24T10:14:10.981 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88700.log 2023-08-24T10:14:10.981 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.87734.log.gz 2023-08-24T10:14:10.981 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91181.log 2023-08-24T10:14:10.981 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85717.log 2023-08-24T10:14:10.981 INFO:teuthology.orchestra.run.smithi060.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.88700.log.gz 2023-08-24T10:14:10.982 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86977.log 2023-08-24T10:14:10.982 INFO:teuthology.orchestra.run.smithi060.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.91181.log.gz 2023-08-24T10:14:10.982 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91223.log 2023-08-24T10:14:10.982 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85717.log: /var/log/ceph/ceph-client.admin.86977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89669.log 2023-08-24T10:14:10.982 INFO:teuthology.orchestra.run.smithi060.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.85717.log.gz 2023-08-24T10:14:10.982 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91223.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.86977.log.gz 2023-08-24T10:14:10.982 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87187.log 2023-08-24T10:14:10.983 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.91223.log.gz 2023-08-24T10:14:10.983 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90131.log 2023-08-24T10:14:10.983 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89669.log: /var/log/ceph/ceph-client.admin.87187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89207.log 2023-08-24T10:14:10.983 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.89669.log.gz 2023-08-24T10:14:10.983 INFO:teuthology.orchestra.run.smithi060.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.87187.log.gz 2023-08-24T10:14:10.983 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84072.log 2023-08-24T10:14:10.983 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.90131.log.gz 2023-08-24T10:14:10.983 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90425.log 2023-08-24T10:14:10.984 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89207.log: /var/log/ceph/ceph-client.admin.84072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86389.log 2023-08-24T10:14:10.984 INFO:teuthology.orchestra.run.smithi060.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.89207.log.gz 2023-08-24T10:14:10.984 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1%/var/log/ceph/ceph-client.admin.90425.log: -- replaced with /var/log/ceph/ceph-client.admin.84072.log.gz 2023-08-24T10:14:10.984 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89963.log 2023-08-24T10:14:10.984 INFO:teuthology.orchestra.run.smithi060.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.90425.log.gz 2023-08-24T10:14:10.984 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91687.log 2023-08-24T10:14:10.984 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86389.log: /var/log/ceph/ceph-client.admin.89963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88406.log 2023-08-24T10:14:10.985 INFO:teuthology.orchestra.run.smithi060.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.86389.log.gz 2023-08-24T10:14:10.985 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.89963.log.gz 2023-08-24T10:14:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90551.log 2023-08-24T10:14:10.985 INFO:teuthology.orchestra.run.smithi060.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.91687.log.gz 2023-08-24T10:14:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86095.log 2023-08-24T10:14:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88406.log: /var/log/ceph/ceph-client.admin.90551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91433.log 2023-08-24T10:14:10.985 INFO:teuthology.orchestra.run.smithi060.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.88406.log.gz 2023-08-24T10:14:10.986 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.90551.log.gz 2023-08-24T10:14:10.986 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90719.log 2023-08-24T10:14:10.986 INFO:teuthology.orchestra.run.smithi060.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.86095.log.gz 2023-08-24T10:14:10.986 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84877.log 2023-08-24T10:14:10.986 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91433.log: /var/log/ceph/ceph-client.admin.90719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86641.log 2023-08-24T10:14:10.986 INFO:teuthology.orchestra.run.smithi060.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.91433.log.gz 2023-08-24T10:14:10.986 INFO:teuthology.orchestra.run.smithi060.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.90719.log.gz 2023-08-24T10:14:10.987 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91518.log 2023-08-24T10:14:10.987 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.84877.log.gz 2023-08-24T10:14:10.987 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87271.log 2023-08-24T10:14:10.987 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86641.log: /var/log/ceph/ceph-client.admin.91518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86179.log 2023-08-24T10:14:10.987 INFO:teuthology.orchestra.run.smithi060.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.86641.log.gz 2023-08-24T10:14:10.987 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.91518.log.gz 2023-08-24T10:14:10.987 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88910.log 2023-08-24T10:14:10.987 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84136.log 2023-08-24T10:14:10.988 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86179.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.87271.log.gz 2023-08-24T10:14:10.988 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88910.log: 90.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-08-24T10:14:10.988 INFO:teuthology.orchestra.run.smithi060.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.86179.log.gz 2023-08-24T10:14:10.988 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89165.log 2023-08-24T10:14:10.988 INFO:teuthology.orchestra.run.smithi060.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.88910.log.gz 2023-08-24T10:14:10.988 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84835.log 2023-08-24T10:14:10.988 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89165.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.84136.log.gz 2023-08-24T10:14:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84180.log 2023-08-24T10:14:10.989 INFO:teuthology.orchestra.run.smithi060.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.89165.log.gz 2023-08-24T10:14:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85003.log 2023-08-24T10:14:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84180.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.84835.log.gz 2023-08-24T10:14:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85549.log 2023-08-24T10:14:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85003.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.84180.log.gz 2023-08-24T10:14:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86851.log 2023-08-24T10:14:10.990 INFO:teuthology.orchestra.run.smithi060.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.85003.log.gz 2023-08-24T10:14:10.990 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90341.log 2023-08-24T10:14:10.990 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.85549.log.gz 2023-08-24T10:14:10.990 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88532.log 2023-08-24T10:14:10.990 INFO:teuthology.orchestra.run.smithi060.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.86851.log.gz 2023-08-24T10:14:10.990 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89081.log 2023-08-24T10:14:10.990 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88532.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.90341.log.gz 2023-08-24T10:14:10.990 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90173.log 2023-08-24T10:14:10.991 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.88532.log.gz 2023-08-24T10:14:10.991 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85171.log 2023-08-24T10:14:10.991 INFO:teuthology.orchestra.run.smithi060.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.89081.log.gz 2023-08-24T10:14:10.991 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88742.log 2023-08-24T10:14:10.991 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85171.log: gzip 85.3% -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.90173.log.gz /var/log/ceph/ceph-client.admin.86263.log 2023-08-24T10:14:10.991 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-08-24T10:14:10.991 INFO:teuthology.orchestra.run.smithi060.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.85171.log.gz 2023-08-24T10:14:10.992 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85675.log 2023-08-24T10:14:10.992 INFO:teuthology.orchestra.run.smithi060.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.88742.log.gz 2023-08-24T10:14:10.992 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85213.log 2023-08-24T10:14:10.992 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.86263.log.gz 2023-08-24T10:14:10.992 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86809.log 2023-08-24T10:14:10.992 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.85675.log.gz 2023-08-24T10:14:10.993 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85213.log: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-08-24T10:14:10.993 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89795.log 2023-08-24T10:14:10.993 INFO:teuthology.orchestra.run.smithi060.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.85213.log.gz 2023-08-24T10:14:10.993 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86347.log 2023-08-24T10:14:10.993 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.86809.log.gz 2023-08-24T10:14:10.993 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84338.log 2023-08-24T10:14:10.993 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.89795.log.gz 2023-08-24T10:14:10.993 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85843.log 2023-08-24T10:14:10.994 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86347.log: /var/log/ceph/ceph-client.admin.84338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88154.log 2023-08-24T10:14:10.994 INFO:teuthology.orchestra.run.smithi060.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.86347.log.gz 2023-08-24T10:14:10.994 INFO:teuthology.orchestra.run.smithi060.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.84338.log.gz 2023-08-24T10:14:10.994 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88028.log 2023-08-24T10:14:10.994 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88154.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.85843.log.gz 2023-08-24T10:14:10.994 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89375.log 2023-08-24T10:14:10.994 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88028.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.88154.log.gz 2023-08-24T10:14:10.995 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84480.log 2023-08-24T10:14:10.995 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.88028.log.gz 2023-08-24T10:14:10.995 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86599.log 2023-08-24T10:14:10.995 INFO:teuthology.orchestra.run.smithi060.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.89375.log.gz 2023-08-24T10:14:10.995 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91603.log 2023-08-24T10:14:10.995 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87818.log 2023-08-24T10:14:10.995 INFO:teuthology.orchestra.run.smithi060.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.84480.log.gz 2023-08-24T10:14:10.995 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91603.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.86599.log.gz 2023-08-24T10:14:10.996 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85633.log 2023-08-24T10:14:10.996 INFO:teuthology.orchestra.run.smithi060.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.91603.log.gz 2023-08-24T10:14:10.996 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91055.log 2023-08-24T10:14:10.996 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85633.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.87818.log.gz 2023-08-24T10:14:10.996 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87355.log 2023-08-24T10:14:10.996 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.85633.log.gz 2023-08-24T10:14:10.997 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90971.log 2023-08-24T10:14:10.997 INFO:teuthology.orchestra.run.smithi060.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.91055.log.gz 2023-08-24T10:14:10.997 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87860.log 2023-08-24T10:14:10.997 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.87355.log.gz 2023-08-24T10:14:10.997 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86725.log 2023-08-24T10:14:10.998 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.90971.log.gz 2023-08-24T10:14:10.998 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86557.log 2023-08-24T10:14:10.998 INFO:teuthology.orchestra.run.smithi060.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.87860.log.gz 2023-08-24T10:14:10.998 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91349.log 2023-08-24T10:14:10.998 INFO:teuthology.orchestra.run.smithi060.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.86725.log.gz 2023-08-24T10:14:10.998 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86557.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-08-24T10:14:10.998 INFO:teuthology.orchestra.run.smithi060.stderr: 86.6%/var/log/ceph/ceph-client.admin.91349.log: -- replaced with /var/log/ceph/ceph-client.admin.86557.log.gz 2023-08-24T10:14:10.998 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89627.log 2023-08-24T10:14:10.999 INFO:teuthology.orchestra.run.smithi060.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.91349.log.gz 2023-08-24T10:14:10.999 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87986.log 2023-08-24T10:14:10.999 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89753.log 2023-08-24T10:14:10.999 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1%/var/log/ceph/ceph-client.admin.87986.log: -- replaced with /var/log/ceph/ceph-client.admin.89627.log.gz 2023-08-24T10:14:10.999 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89879.log 2023-08-24T10:14:10.999 INFO:teuthology.orchestra.run.smithi060.stderr: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-08-24T10:14:10.999 INFO:teuthology.orchestra.run.smithi060.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.87986.log.gz 2023-08-24T10:14:11.000 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86011.log 2023-08-24T10:14:11.000 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89879.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.89753.log.gz 2023-08-24T10:14:11.000 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84482.log 2023-08-24T10:14:11.000 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.89879.log.gz 2023-08-24T10:14:11.000 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87145.log 2023-08-24T10:14:11.000 INFO:teuthology.orchestra.run.smithi060.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.86011.log.gz 2023-08-24T10:14:11.000 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88196.log 2023-08-24T10:14:11.001 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86137.log 2023-08-24T10:14:11.001 INFO:teuthology.orchestra.run.smithi060.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.84482.log.gz 2023-08-24T10:14:11.001 INFO:teuthology.orchestra.run.smithi060.stderr: 86.3%/var/log/ceph/ceph-client.admin.88196.log: -- replaced with /var/log/ceph/ceph-client.admin.87145.log.gz 2023-08-24T10:14:11.001 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89921.log 2023-08-24T10:14:11.001 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.88196.log.gz 2023-08-24T10:14:11.001 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86515.log 2023-08-24T10:14:11.002 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89921.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.86137.log.gz 2023-08-24T10:14:11.002 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86053.log 2023-08-24T10:14:11.002 INFO:teuthology.orchestra.run.smithi060.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.89921.log.gz 2023-08-24T10:14:11.002 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84751.log 2023-08-24T10:14:11.002 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.86515.log.gz 2023-08-24T10:14:11.003 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91139.log 2023-08-24T10:14:11.003 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86053.log: /var/log/ceph/ceph-client.admin.84751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90593.log 2023-08-24T10:14:11.003 INFO:teuthology.orchestra.run.smithi060.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.86053.log.gz 2023-08-24T10:14:11.003 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91139.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.84751.log.gz 2023-08-24T10:14:11.003 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91013.log 2023-08-24T10:14:11.003 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.91139.log.gz 2023-08-24T10:14:11.003 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85297.log 2023-08-24T10:14:11.004 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90593.log: /var/log/ceph/ceph-client.admin.91013.log: gzip -5 86.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.90593.log.gz 2023-08-24T10:14:11.004 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.90467.log 2023-08-24T10:14:11.004 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.91013.log.gz 2023-08-24T10:14:11.004 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89333.log 2023-08-24T10:14:11.004 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86221.log 2023-08-24T10:14:11.004 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90467.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.85297.log.gz 2023-08-24T10:14:11.004 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89837.log 2023-08-24T10:14:11.005 INFO:teuthology.orchestra.run.smithi060.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.90467.log.gz 2023-08-24T10:14:11.005 INFO:teuthology.orchestra.run.smithi060.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.89333.log.gz 2023-08-24T10:14:11.005 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85339.log 2023-08-24T10:14:11.005 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89837.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.86221.log.gz 2023-08-24T10:14:11.005 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90215.log 2023-08-24T10:14:11.005 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85339.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.89837.log.gz 2023-08-24T10:14:11.005 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87607.log 2023-08-24T10:14:11.006 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.85339.log.gz 2023-08-24T10:14:11.006 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89417.log 2023-08-24T10:14:11.006 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.90215.log.gz 2023-08-24T10:14:11.006 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87523.log 2023-08-24T10:14:11.006 INFO:teuthology.orchestra.run.smithi060.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.87607.log.gz 2023-08-24T10:14:11.006 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88616.log 2023-08-24T10:14:11.006 INFO:teuthology.orchestra.run.smithi060.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.89417.log.gz 2023-08-24T10:14:11.007 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91855.log 2023-08-24T10:14:11.007 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.87523.log.gz 2023-08-24T10:14:11.007 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84380.log 2023-08-24T10:14:11.007 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.88616.log.gz 2023-08-24T10:14:11.007 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84793.log 2023-08-24T10:14:11.007 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85591.log 2023-08-24T10:14:11.008 INFO:teuthology.orchestra.run.smithi060.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.84380.log.gz 2023-08-24T10:14:11.008 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88995.log 2023-08-24T10:14:11.008 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.84793.log.gz 2023-08-24T10:14:11.008 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87019.log 2023-08-24T10:14:11.008 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88995.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.85591.log.gz 2023-08-24T10:14:11.008 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88952.log 2023-08-24T10:14:11.009 INFO:teuthology.orchestra.run.smithi060.stderr: 96.1% -- replaced with /var/log/ceph/ceph-client.admin.91855.log.gz 2023-08-24T10:14:11.009 INFO:teuthology.orchestra.run.smithi060.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.88995.log.gz 2023-08-24T10:14:11.009 INFO:teuthology.orchestra.run.smithi060.stderr:gzip/var/log/ceph/ceph-client.admin.87019.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.88070.log 2023-08-24T10:14:11.009 INFO:teuthology.orchestra.run.smithi060.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.87019.log.gz 2023-08-24T10:14:11.009 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87650.log 2023-08-24T10:14:11.009 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88070.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.88952.log.gzgzip 2023-08-24T10:14:11.009 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.88658.log 2023-08-24T10:14:11.009 INFO:teuthology.orchestra.run.smithi060.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.88070.log.gz 2023-08-24T10:14:11.010 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88322.log 2023-08-24T10:14:11.010 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89123.log 2023-08-24T10:14:11.010 INFO:teuthology.orchestra.run.smithi060.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.87650.log.gz 2023-08-24T10:14:11.010 INFO:teuthology.orchestra.run.smithi060.stderr:gzip 86.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.88658.log.gz --verbose 2023-08-24T10:14:11.010 INFO:teuthology.orchestra.run.smithi060.stderr: --/var/log/ceph/ceph-client.admin.88322.log: /var/log/ceph/ceph-client.admin.88490.log 2023-08-24T10:14:11.010 INFO:teuthology.orchestra.run.smithi060.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.88322.log.gz 2023-08-24T10:14:11.011 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85465.log 2023-08-24T10:14:11.011 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84280.log 86.2% 2023-08-24T10:14:11.011 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.89123.log.gz 2023-08-24T10:14:11.011 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85465.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.88490.log.gz 2023-08-24T10:14:11.011 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84961.log 2023-08-24T10:14:11.011 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.85465.log.gz 2023-08-24T10:14:11.012 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88238.log 2023-08-24T10:14:11.012 INFO:teuthology.orchestra.run.smithi060.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.84280.log.gz 2023-08-24T10:14:11.012 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86683.log 2023-08-24T10:14:11.012 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88238.log: 85.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.84961.log.gz 2023-08-24T10:14:11.012 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.87481.log 2023-08-24T10:14:11.013 INFO:teuthology.orchestra.run.smithi060.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.88238.log.gz 2023-08-24T10:14:11.013 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87103.log 2023-08-24T10:14:11.013 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.86683.log.gz 2023-08-24T10:14:11.013 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87061.log 2023-08-24T10:14:11.013 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87481.log: /var/log/ceph/ceph-client.admin.87103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83228.log 2023-08-24T10:14:11.013 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.87481.log.gz 2023-08-24T10:14:11.013 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.87103.log.gz 2023-08-24T10:14:11.014 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91771.log 2023-08-24T10:14:11.014 INFO:teuthology.orchestra.run.smithi060.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.87061.log.gz 2023-08-24T10:14:11.014 INFO:teuthology.orchestra.run.smithi060.stderr:gzip/var/log/ceph/ceph-client.admin.83228.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.91391.log 2023-08-24T10:14:11.014 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91771.log: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.83228.log.gz 2023-08-24T10:14:11.014 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91813.log 2023-08-24T10:14:11.014 INFO:teuthology.orchestra.run.smithi060.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.91771.log.gz 2023-08-24T10:14:11.014 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85969.log 2023-08-24T10:14:11.014 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.91391.log.gz 2023-08-24T10:14:11.015 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85423.log 2023-08-24T10:14:11.015 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.91813.log.gz 2023-08-24T10:14:11.015 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85801.log 2023-08-24T10:14:11.015 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86431.log 2023-08-24T10:14:11.015 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85423.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.85969.log.gz 2023-08-24T10:14:11.015 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.85423.log.gz 2023-08-24T10:14:11.015 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85759.log 2023-08-24T10:14:11.015 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86431.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.85801.log.gz 2023-08-24T10:14:11.016 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91729.log 2023-08-24T10:14:11.016 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.86431.log.gz 2023-08-24T10:14:11.016 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84479.log 2023-08-24T10:14:11.016 INFO:teuthology.orchestra.run.smithi060.stderr: 86.2%/var/log/ceph/ceph-client.admin.91729.log: -- replaced with /var/log/ceph/ceph-client.admin.85759.log.gz 2023-08-24T10:14:11.016 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90089.log 2023-08-24T10:14:11.016 INFO:teuthology.orchestra.run.smithi060.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.91729.log.gz 2023-08-24T10:14:11.016 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91097.log 2023-08-24T10:14:11.016 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88826.log 90.3% 2023-08-24T10:14:11.017 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.84479.log.gz 2023-08-24T10:14:11.017 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91097.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.90089.log.gz 2023-08-24T10:14:11.017 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87229.log 2023-08-24T10:14:11.017 INFO:teuthology.orchestra.run.smithi060.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.91097.log.gz 2023-08-24T10:14:11.017 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88868.log 2023-08-24T10:14:11.017 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91475.log 2023-08-24T10:14:11.017 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.88826.log.gz 2023-08-24T10:14:11.018 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.87229.log.gz 2023-08-24T10:14:11.018 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89249.log 2023-08-24T10:14:11.018 INFO:teuthology.orchestra.run.smithi060.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.88868.log.gz 2023-08-24T10:14:11.018 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82993.log 2023-08-24T10:14:11.018 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89249.log: gzip 85.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.91475.log.gz 2023-08-24T10:14:11.019 INFO:teuthology.orchestra.run.smithi060.stderr: -- /var/log/ceph/ceph-client.admin.90929.log 2023-08-24T10:14:11.019 INFO:teuthology.orchestra.run.smithi060.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.89249.log.gz 2023-08-24T10:14:11.019 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.82993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87692.log 2023-08-24T10:14:11.019 INFO:teuthology.orchestra.run.smithi060.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.82993.log.gz 2023-08-24T10:14:11.019 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88784.log 2023-08-24T10:14:11.019 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90929.log: /var/log/ceph/ceph-client.admin.87692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86935.log 2023-08-24T10:14:11.019 INFO:teuthology.orchestra.run.smithi060.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.90929.log.gz 2023-08-24T10:14:11.019 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88784.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.87692.log.gz 2023-08-24T10:14:11.020 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-08-24T10:14:11.020 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.88784.log.gz 2023-08-24T10:14:11.020 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91885.log 2023-08-24T10:14:11.020 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.86935.log.gz 2023-08-24T10:14:11.021 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87439.log 2023-08-24T10:14:11.021 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84667.log 2023-08-24T10:14:11.021 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph.log.gz 2023-08-24T10:14:11.021 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90803.log 2023-08-24T10:14:11.021 INFO:teuthology.orchestra.run.smithi060.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.87439.log.gz 2023-08-24T10:14:11.022 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90635.log 2023-08-24T10:14:11.022 INFO:teuthology.orchestra.run.smithi060.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.84667.log.gz 2023-08-24T10:14:11.022 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86473.log 2023-08-24T10:14:11.022 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90803.log: /var/log/ceph/ceph-client.admin.90635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87902.log 2023-08-24T10:14:11.022 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.90803.log.gz 2023-08-24T10:14:11.023 INFO:teuthology.orchestra.run.smithi060.stderr: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.91885.log.gz 2023-08-24T10:14:11.023 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86473.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.90635.log.gz 2023-08-24T10:14:11.023 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87397.log 2023-08-24T10:14:11.023 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.86473.log.gz 2023-08-24T10:14:11.023 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85255.log 2023-08-24T10:14:11.023 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.87902.log.gz 2023-08-24T10:14:11.023 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87397.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-08-24T10:14:11.024 INFO:teuthology.orchestra.run.smithi060.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.87397.log.gz 2023-08-24T10:14:11.024 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85255.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.80685.log 2023-08-24T10:14:11.024 INFO:teuthology.orchestra.run.smithi060.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.85255.log.gz 2023-08-24T10:14:11.024 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89291.log 2023-08-24T10:14:11.024 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph.tmp-client.admin.80685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90005.log 2023-08-24T10:14:11.024 INFO:teuthology.orchestra.run.smithi060.stderr: 66.4% -- replaced with /var/log/ceph/ceph.tmp-client.admin.80685.log.gz 2023-08-24T10:14:11.025 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85507.log 2023-08-24T10:14:11.025 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.89291.log.gz 2023-08-24T10:14:11.025 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85927.log 2023-08-24T10:14:11.025 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90005.log: /var/log/ceph/ceph-client.admin.85507.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.90005.log.gz 2023-08-24T10:14:11.025 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85885.log 2023-08-24T10:14:11.025 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85927.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.85507.log.gz 2023-08-24T10:14:11.025 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89543.log 2023-08-24T10:14:11.025 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.85927.log.gz 2023-08-24T10:14:11.026 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.85885.log: /var/log/ceph/ceph-client.admin.86305.log 2023-08-24T10:14:11.026 INFO:teuthology.orchestra.run.smithi060.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.85885.log.gz 2023-08-24T10:14:11.026 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87313.log 2023-08-24T10:14:11.026 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86305.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.89543.log.gz 2023-08-24T10:14:11.026 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90887.log 2023-08-24T10:14:11.026 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.86305.log.gz 2023-08-24T10:14:11.026 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85129.log 2023-08-24T10:14:11.026 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.87313.log.gz 2023-08-24T10:14:11.027 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90383.log 2023-08-24T10:14:11.027 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.90887.log.gz 2023-08-24T10:14:11.027 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85045.log 2023-08-24T10:14:11.027 INFO:teuthology.orchestra.run.smithi060.stderr: 86.2%/var/log/ceph/ceph-client.admin.90383.log: -- replaced with /var/log/ceph/ceph-client.admin.85129.log.gz 2023-08-24T10:14:11.027 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90761.log 2023-08-24T10:14:11.027 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85045.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.90383.log.gz 2023-08-24T10:14:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84625.log 2023-08-24T10:14:11.028 INFO:teuthology.orchestra.run.smithi060.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.85045.log.gz 2023-08-24T10:14:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86893.log 2023-08-24T10:14:11.028 INFO:teuthology.orchestra.run.smithi060.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.90761.log.gz 2023-08-24T10:14:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85381.log 2023-08-24T10:14:11.028 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86893.log: gzip -5 --verbose 85.9% -- /var/log/ceph/ceph-client.admin.88112.log -- replaced with /var/log/ceph/ceph-client.admin.84625.log.gz 2023-08-24T10:14:11.028 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-08-24T10:14:11.029 INFO:teuthology.orchestra.run.smithi060.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.86893.log.gz 2023-08-24T10:14:11.029 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91561.log 2023-08-24T10:14:11.029 INFO:teuthology.orchestra.run.smithi060.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.85381.log.gz 2023-08-24T10:14:11.029 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90677.log 2023-08-24T10:14:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88112.log: /var/log/ceph/ceph-client.admin.91561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89711.log 2023-08-24T10:14:11.030 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.88112.log.gz 2023-08-24T10:14:11.030 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.91561.log.gz 2023-08-24T10:14:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86767.log 2023-08-24T10:14:11.030 INFO:teuthology.orchestra.run.smithi060.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.90677.log.gz 2023-08-24T10:14:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89459.log 2023-08-24T10:14:11.030 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86767.log: 85.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.89711.log.gz -5 2023-08-24T10:14:11.030 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.84919.log 2023-08-24T10:14:11.031 INFO:teuthology.orchestra.run.smithi060.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.86767.log.gz 2023-08-24T10:14:11.031 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87565.log 2023-08-24T10:14:11.031 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.89459.log.gz/var/log/ceph/ceph-client.admin.84919.log: 2023-08-24T10:14:11.031 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84709.log 2023-08-24T10:14:11.031 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87565.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.84919.log.gz 2023-08-24T10:14:11.031 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83256.log 2023-08-24T10:14:11.032 INFO:teuthology.orchestra.run.smithi060.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.87565.log.gz 2023-08-24T10:14:11.032 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89501.log 2023-08-24T10:14:11.032 INFO:teuthology.orchestra.run.smithi060.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.84709.log.gz 2023-08-24T10:14:11.032 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88448.log 2023-08-24T10:14:11.032 INFO:teuthology.orchestra.run.smithi060.stderr: 84.7%/var/log/ceph/ceph-client.admin.89501.log: -- replaced with /var/log/ceph/ceph-client.admin.83256.log.gz 2023-08-24T10:14:11.032 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.89501.log.gz 2023-08-24T10:14:11.033 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88448.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.88448.log.gz 2023-08-24T10:14:11.080 INFO:teuthology.orchestra.run.smithi060.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-08-24T10:14:11.082 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-08-24T10:14:11.082 INFO:teuthology.orchestra.run.smithi060.stderr:real 0m0.177s 2023-08-24T10:14:11.082 INFO:teuthology.orchestra.run.smithi060.stderr:user 0m0.235s 2023-08-24T10:14:11.082 INFO:teuthology.orchestra.run.smithi060.stderr:sys 0m0.095s 2023-08-24T10:14:11.083 INFO:tasks.ceph:Archiving logs... 2023-08-24T10:14:11.083 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/log/ceph to /home/teuthworker/archive/matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi/7378498/remote/smithi060/log 2023-08-24T10:14:11.085 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-24T10:14:11.337 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/log/ceph to /home/teuthworker/archive/matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi/7378498/remote/smithi135/log 2023-08-24T10:14:11.339 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-24T10:14:11.416 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1938, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1496, in healthy manager.wait_for_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2743, in wait_for_clean while not self.is_clean(): File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2679, in is_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-08-24T10:14:11.490 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=08fe4c8e7db34dc29c323d8862d0d342 Traceback (most recent call last): File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1938, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1496, in healthy manager.wait_for_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2743, in wait_for_clean while not self.is_clean(): File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2679, in is_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_Matan-B_ceph_74f8c0ae3720a6c1aeeb00acc38a54b1cf5fdabe/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-08-24T10:14:11.497 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-08-24T10:14:11.518 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-08-24T10:14:11.527 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-08-24T10:14:11.527 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-24T10:14:11.540 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-24T10:14:11.621 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-08-24T10:14:11.631 INFO:teuthology.task.clock:Checking final clock skew... 2023-08-24T10:14:11.631 DEBUG:teuthology.orchestra.run.smithi060:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-24T10:14:11.634 DEBUG:teuthology.orchestra.run.smithi135:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-24T10:14:11.646 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2023-08-24T10:14:11.649 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2023-08-24T10:14:11.650 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-24T10:14:11.650 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2023-08-24T10:14:11.651 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 64 +2433us[+2675us] +/- 73ms 2023-08-24T10:14:11.651 INFO:teuthology.orchestra.run.smithi060.stdout:^* hv02.front.sepia.ceph.com 3 6 377 64 -2213us[-1971us] +/- 68ms 2023-08-24T10:14:11.651 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 +161us[ +161us] +/- 68ms 2023-08-24T10:14:11.651 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-08-24T10:14:11.653 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-24T10:14:11.654 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2023-08-24T10:14:11.654 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 1 +2435us[+2444us] +/- 74ms 2023-08-24T10:14:11.654 INFO:teuthology.orchestra.run.smithi135.stdout:^* hv02.front.sepia.ceph.com 3 6 377 1 -2224us[-2215us] +/- 69ms 2023-08-24T10:14:11.654 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 +146us[ +149us] +/- 68ms 2023-08-24T10:14:11.654 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-08-24T10:14:11.655 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-08-24T10:14:11.665 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-08-24T10:14:11.666 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-08-24T10:14:11.674 DEBUG:teuthology.orchestra.run.smithi060:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-24T10:14:11.797 DEBUG:teuthology.orchestra.run.smithi135:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-24T10:14:11.892 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-24T10:14:11.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-24T10:14:11.929 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-24T10:14:11.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-24T10:14:11.962 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-08-24T10:14:11.972 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-08-24T10:14:11.981 INFO:teuthology.task.internal:Duration was 1191.846233 seconds 2023-08-24T10:14:11.982 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-08-24T10:14:11.995 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-08-24T10:14:11.996 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-24T10:14:11.998 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-24T10:14:12.046 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-24T10:14:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-24T10:14:12.382 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-08-24T10:14:12.382 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi060.front.sepia.ceph.com 2023-08-24T10:14:12.383 DEBUG:teuthology.orchestra.run.smithi060:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-24T10:14:12.405 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi135.front.sepia.ceph.com 2023-08-24T10:14:12.406 DEBUG:teuthology.orchestra.run.smithi135:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-24T10:14:12.428 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-08-24T10:14:12.429 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-24T10:14:12.447 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-24T10:14:12.497 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-08-24T10:14:12.497 DEBUG:teuthology.orchestra.run.smithi060:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-24T10:14:12.519 DEBUG:teuthology.orchestra.run.smithi135:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-24T10:14:13.012 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-08-24T10:14:13.023 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-08-24T10:14:13.023 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-24T10:14:13.054 DEBUG:teuthology.orchestra.run.smithi135:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-24T10:14:13.081 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-08-24T10:14:13.098 DEBUG:teuthology.orchestra.run.smithi060:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-24T10:14:13.101 DEBUG:teuthology.orchestra.run.smithi135:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-24T10:14:13.128 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = core 2023-08-24T10:14:13.147 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = core 2023-08-24T10:14:13.214 DEBUG:teuthology.orchestra.run.smithi060:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-24T10:14:13.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-24T10:14:13.229 DEBUG:teuthology.orchestra.run.smithi135:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-24T10:14:13.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-24T10:14:13.268 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-08-24T10:14:13.278 INFO:teuthology.task.internal:Transferring archived files... 2023-08-24T10:14:13.278 DEBUG:teuthology.misc:Transferring archived files from smithi060:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi/7378498/remote/smithi060 2023-08-24T10:14:13.279 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-24T10:14:13.319 DEBUG:teuthology.misc:Transferring archived files from smithi135:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi/7378498/remote/smithi135 2023-08-24T10:14:13.319 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-24T10:14:13.356 INFO:teuthology.task.internal:Removing archive directory... 2023-08-24T10:14:13.356 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-24T10:14:13.359 DEBUG:teuthology.orchestra.run.smithi135:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-24T10:14:13.403 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-08-24T10:14:13.413 INFO:teuthology.task.internal:Not uploading archives. 2023-08-24T10:14:13.413 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-08-24T10:14:13.459 INFO:teuthology.task.internal:Tidying up after the test... 2023-08-24T10:14:13.459 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-24T10:14:13.462 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-24T10:14:13.474 INFO:teuthology.orchestra.run.smithi060.stdout: 263918 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 24 10:14 /home/ubuntu/cephtest 2023-08-24T10:14:13.476 INFO:teuthology.orchestra.run.smithi135.stdout: 263626 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 24 10:14 /home/ubuntu/cephtest 2023-08-24T10:14:13.477 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-08-24T10:14:13.491 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-08-24T10:14:13.520 INFO:teuthology.nuke:Checking targets against current locks 2023-08-24T10:14:13.547 DEBUG:teuthology.nuke:shortname: smithi060 2023-08-24T10:14:13.547 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-24T10:14:13.571 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi/7378498', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-24 09:46:20.439239', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN6gNUKYLjbh09BzRantQjCUQIPym0AcypVmYdTq0EQhIUlhNLQG2LbakWKji0BnsgZy+jdMt47mxajWjPvBTFc='} 2023-08-24T10:14:13.575 DEBUG:teuthology.nuke:shortname: smithi135 2023-08-24T10:14:13.575 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-24T10:14:13.597 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2023-08-24_09:25:19-crimson-rados-wip-matanb-crimson-testing-24.8-v2-distro-crimson-smithi/7378498', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-24 09:46:20.437484', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLiGtAHJjeXgDa3WaBv86LmWFjgoBL171vZvf/ibHaA9UP1RgxslYuGqz4owpEQtVYvE+83r0dcAeW86D74iIBc='} 2023-08-24T10:14:13.628 INFO:teuthology.orchestra.console.smithi060:Power off 2023-08-24T10:14:13.630 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-24T10:14:13.655 DEBUG:teuthology.orchestra.console.smithi060:power off output: Chassis Power Control: Down/Off 2023-08-24T10:14:13.656 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-24T10:14:13.673 INFO:teuthology.orchestra.console.smithi135:Power off 2023-08-24T10:14:13.674 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-24T10:14:13.685 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2023-08-24T10:14:13.698 DEBUG:teuthology.orchestra.console.smithi135:power off output: Chassis Power Control: Down/Off 2023-08-24T10:14:13.699 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-24T10:14:13.722 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is on 2023-08-24T10:14:17.687 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-24T10:14:17.723 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-24T10:14:17.800 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2023-08-24T10:14:17.836 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is on 2023-08-24T10:14:21.804 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-24T10:14:21.837 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-24T10:14:21.920 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is off 2023-08-24T10:14:21.949 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is off 2023-08-24T10:14:22.021 INFO:teuthology.orchestra.console.smithi060:Power off completed 2023-08-24T10:14:22.051 INFO:teuthology.orchestra.console.smithi135:Power off completed 2023-08-24T10:14:22.155 INFO:teuthology.run:Summary data: description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} duration: 1191.8462331295013 failure_reason: 'Command failed on smithi060 with status 124: ''sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json''' flavor: crimson owner: scheduled_matan@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=08fe4c8e7db34dc29c323d8862d0d342 status: fail success: false 2023-08-24T10:14:22.156 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-24T10:14:22.226 INFO:teuthology.run:FAIL